Hi,
I'm using following Avro Parquet writer to convert our Avro to Parquet:
writer = AvroParquetWriter
.builder( new Path( parquetFile.getPath() ))
.withSchema(schema)
.enableDictionaryEncoding()
.withDataModel(ReflectData.get())
.withWriterVersion(ParquetProperties.WriterVersion.PARQUET_2_0)
//.withCompressionCodec(CompressionCodecName.SNAPPY)
.withDictionaryPageSize(2*1024*1024)
.build();
It works well without the compression but if I turn it on then drill hangs
on any queries and does not recover.
Does anyone have experience in this or ides on why this is happening?
Regards,
-Stefán
|