drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefán Baxter <ste...@activitystream.com>
Subject Issue with compression :: Using Drill 1.5 and parquet-mr/parquet-avro 1.8.1
Date Mon, 08 Feb 2016 20:37:07 GMT
Hi,

I'm using following Avro Parquet writer to convert our Avro to Parquet:

writer = AvroParquetWriter

        .builder( new Path( parquetFile.getPath() ))
        .withSchema(schema)
        .enableDictionaryEncoding()
        .withDataModel(ReflectData.get())
        .withWriterVersion(ParquetProperties.WriterVersion.PARQUET_2_0)
        //.withCompressionCodec(CompressionCodecName.SNAPPY)
        .withDictionaryPageSize(2*1024*1024)
        .build();

It works well without the compression but if I turn it on then drill hangs
on any queries and does not recover.

Does anyone have experience in this or ides on why this is happening?

Regards,
 -Stefán

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message