drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Edike <peter.ed...@interswitchgroup.com>
Subject Error Querying Parquet Files In Apache Drill 1.13
Date Tue, 07 Aug 2018 16:10:15 GMT
Hello everyone ,

Please I am having issues querying some parquet files generated using scoop1 in Apache Drill.

I checked the logs and I see the following exception everywhere

Aug 7, 2018 4:28:09 PM WARNING: org.apache.parquet.CorruptStatistics: Ignoring statistics
because created_by could not be parsed (see PARQUET-251): parquet-mr (build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb)
org.apache.parquet.VersionParser$VersionParseException: Could not parse created_by: parquet-mr
(build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb) using format: (.+) version ((.*) )?\(build
?(.*)\)
        at org.apache.parquet.VersionParser.parse(VersionParser.java:112)
        at org.apache.parquet.CorruptStatistics.shouldIgnoreStatistics(CorruptStatistics.java:66)
        at org.apache.parquet.format.converter.ParquetMetadataConverter.fromParquetStatistics(ParquetMetadataConverter.java:264)
        at org.apache.parquet.format.converter.ParquetMetadataConverter.fromParquetMetadata(ParquetMetadataConverter.java:568)
        at org.apache.parquet.format.converter.ParquetMetadataConverter.readParquetMetadata(ParquetMetadataConverter.java:545)
        at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:455)
        at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:412)
        at org.apache.drill.exec.store.parquet.Metadata$1.run(Metadata.java:435)
        at org.apache.drill.exec.store.parquet.Metadata$1.run(Metadata.java:428)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1633)
        at org.apache.drill.exec.store.parquet.Metadata.getParquetFileMetadata_v3(Metadata.java:428)
        at org.apache.drill.exec.store.parquet.Metadata.access$100(Metadata.java:96)
        at org.apache.drill.exec.store.parquet.Metadata$MetadataGatherer.runInner(Metadata.java:364)
        at org.apache.drill.exec.store.parquet.Metadata$MetadataGatherer.runInner(Metadata.java:352)
        at org.apache.drill.exec.store.TimedRunnable.run(TimedRunnable.java:56)
        at org.apache.drill.exec.store.TimedRunnable$LatchedRunnable.run(TimedRunnable.java:98)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Please what can I do to mitigate against this

________________________________
Peter Edike
Senior Software Engineer
Research and Development, ENG
Engineering
[cid:image002.png@01D42E71.8D117260]
Office  NO:
Mobile NO:
Email: peter.edike@interswitchgroup.com<mailto:peter.edike@interswitchgroup.com>
Interswitch Limited
1648C Oko-Awo Street, Victoria Island Lagos
Customer Contact Centre 0700-9065000
? http://www.interswitchgroup.com<http://www.interswitchgroup.com/>
[cid:image003.png@01D42E71.8D117260]<https://www.quickteller.com/loan-request>

This e-mail and all  attachments transmitted with it remain the property of Interswitch Limited
, the information contained herein  are private  confidential and intended solely for the
use of the addressee. If you have received this e-mail in error, kindly notify the sender.
If you are not the addressee, you should not disseminate, distribute or copy this e-mail.
Kindly notify Interswitch immediately by email if you have received this email in error and
delete this email and any attachment from your system  Emails cannot be guaranteed to be secure
or error free as the message and any attachments could be intercepted, corrupted, lost, delayed,
incomplete or amended. the contents of this email or its attachments have been scanned for
all viruses and all reasonable measures have been taken to ensure that no viruses are present.
Interswitch Limited and its subsidiaries do not accept liability for damage caused by this
email or any attachments.This message has been marked as CONFIDENTIAL on Tuesday, August 7,
2018 @ 5:10:31 PM


Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message