drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From PROJJWAL SAHA <proj.s...@gmail.com>
Subject Re: Exception while reading parquet data
Date Thu, 12 Oct 2017 08:19:37 GMT
sure, I can try disabling sync parquet reader.
Will this however, impact the performance of queries on parquet data ?

On Thu, Oct 12, 2017 at 9:39 AM, Kunal Khatua <kkhatua@mapr.com> wrote:

> If this resolves the issue, could you share some additional details, such
> as the metadata of the Parquet files, the OS, etc.? Details describing the
> setup is also very helpful in identifying what could be the cause of the
> error.
>
> We had observed some similar DATA_READ errors in the early iterations of
> the Async Parquet reader, but those have been resolved. I'm presuming
> you're already on the latest (i.e. Apache Drill 1.11.0)
>
> -----Original Message-----
> From: Arjun kr [mailto:arjun.kr@outlook.com]
> Sent: Wednesday, October 11, 2017 6:52 PM
> To: user@drill.apache.org
> Subject: Re: Exception while reading parquet data
>
>
> Can you try disabling async parquet reader to see if problem gets resolved.
>
>
> alter session set `store.parquet.reader.pagereader.async`=false;
>
> Thanks,
>
> Arjun
>
>
> ________________________________
> From: PROJJWAL SAHA <proj.saha@gmail.com>
> Sent: Wednesday, October 11, 2017 2:20 PM
> To: user@drill.apache.org
> Subject: Exception while reading parquet data
>
> I get below exception when querying parquet data on Oracle Storage Cloud
> service.
> Any pointers on what does this point to ?
>
> Regards,
> Projjwal
>
>
> ERROR o.a.d.e.u.f.BufferedDirectBufInputStream - Error reading from
> stream part-00006-25a9ae4b-fd9e-4770-b17e-9a29b270a4c2.parquet. Error was
> : null
> 2017-10-09 09:42:18,516 [scan-2] INFO  o.a.d.e.s.p.c.AsyncPageReader -
> User Error Occurred: Exception occurred while reading from disk.
> (java.lang.IndexOutOfBoundsException)
> org.apache.drill.common.exceptions.UserException: DATA_READ ERROR:
> Exception occurred while reading from disk.
>
> File:
> /data25GB/storereturns/part-00006-25a9ae4b-fd9e-4770-b17e-
> 9a29b270a4c2.parquet
> Column:  sr_return_time_sk
> Row Group Start:  479751
>
> [Error Id: 10680bb8-d1d6-43a1-b5e0-ef15bd8a9406 ] at
> org.apache.drill.common.exceptions.UserException$
> Builder.build(UserException.java:550)
> ~[drill-common-1.11.0.jar:1.11.0]
> at
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.
> handleAndThrowException(AsyncPageReader.java:185)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at
> org.apache.drill.exec.store.parquet.columnreaders.
> AsyncPageReader.access$700(AsyncPageReader.java:82)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$
> AsyncPageReaderTask.call(AsyncPageReader.java:461)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$
> AsyncPageReaderTask.call(AsyncPageReader.java:381)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_121] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] Caused by:
> java.io.IOException: java.lang.IndexOutOfBoundsException
> at
> org.apache.drill.exec.util.filereader.BufferedDirectBufInputStream.
> getNextBlock(BufferedDirectBufInputStream.java:185)
> ~[drill-java-exec-1.11.0.jar:1.11.0]
> at
> org.apache.drill.exec.util.filereader.BufferedDirectBufInputStream.
> readInternal(BufferedDirectBufInputStream.java:212)
> ~[drill-java-exec-1.11.0.jar:1.11.0]
> at
> org.apache.drill.exec.util.filereader.BufferedDirectBufInputStream.read(
> BufferedDirectBufInputStream.java:277)
> ~[drill-java-exec-1.11.0.jar:1.11.0]
> at
> org.apache.drill.exec.util.filereader.DirectBufInputStream.getNext(
> DirectBufInputStream.java:111)
> ~[drill-java-exec-1.11.0.jar:1.11.0]
> at
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$
> AsyncPageReaderTask.call(AsyncPageReader.java:421)
> [drill-java-exec-1.11.0.jar:1.11.0]
> ... 5 common frames omitted
> Caused by: java.lang.IndexOutOfBoundsException: null at
> java.nio.Buffer.checkBounds(Buffer.java:567) ~[na:1.8.0_121] at
> java.nio.ByteBuffer.put(ByteBuffer.java:827) ~[na:1.8.0_121] at
> java.nio.DirectByteBuffer.put(DirectByteBuffer.java:379) ~[na:1.8.0_121]
> at
> org.apache.parquet.hadoop.util.CompatibilityUtil.getBuf(
> CompatibilityUtil.java:110)
> ~[parquet-hadoop-1.8.1-drill-r0.jar:1.8.1-drill-r0]
> at
> org.apache.drill.exec.util.filereader.BufferedDirectBufInputStream.
> getNextBlock(BufferedDirectBufInputStream.java:182)
> ~[drill-java-exec-1.11.0.jar:1.11.0]
> ... 9 common frames omitted
> 2017-10-09 09:42:20,533 [26248359-2fc8-d177-c3a6-507f6857e0ea:frag:2:3]
> INFO  o.a.d.e.w.fragment.FragmentExecutor -
> 26248359-2fc8-d177-c3a6-507f6857e0ea:2:3: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2017-10-09 09:42:20,533 [26248359-2fc8-d177-c3a6-507f6857e0ea:frag:2:3]
> INFO  o.a.d.e.w.f.FragmentStatusReporter -
> 26248359-2fc8-d177-c3a6-507f6857e0ea:2:3: State to report: RUNNING
> 2017-10-09 09:42:20,534 [26248359-2fc8-d177-c3a6-507f6857e0ea:frag:2:3]
> INFO  o.a.d.e.w.fragment.FragmentExecutor -
> 26248359-2fc8-d177-c3a6-507f6857e0ea:2:3: State change requested RUNNING
> --> CANCELLATION_REQUESTED
> 2017-10-09 09:42:20,534 [26248359-2fc8-d177-c3a6-507f6857e0ea:frag:2:3]
> INFO  o.a.d.e.w.f.FragmentStatusReporter -
> 26248359-2fc8-d177-c3a6-507f6857e0ea:2:3: State to report:
> CANCELLATION_REQUESTED
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message