drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From François Méthot <fmetho...@gmail.com>
Subject Query fails on corrupted parquet column
Date Tue, 02 Feb 2016 17:40:26 GMT

Using drill-embedded 1.4, I encountered this error while doing a query on
folders containing thousands of parquet files:


Fragment 1:9

After re-running the same query with the log level set to DEBUG, I tracked
the files that were scanned by Fragment 1:9, performed the same query on
each individual file until I got the same error.

It turned out that a column in one of the parquet file is causing this
issue. Whether it is an issue with our parquet writer or with the drill
reader remains to be determined.

My questions is :

Is there an option to have a fragment thread to move on to the next file
after it encounter such error, without completely spoiling the whole query
and result?

Also in this case, it would have been useful if it was clearly specified in
the log which parquet file is causing issue.

Thanks a lot


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message