drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From PROJJWAL SAHA <proj.s...@gmail.com>
Subject Re: Exception when querying parquet data
Date Wed, 11 Oct 2017 08:46:41 GMT
I am using Oracle storage cloud service.
I did not run refresh table metadata. Even refresh table metadata fails
with this message.

I think it depends on external network . One another day, the same query
works fine..

Regards,
Projjwal

On Mon, Oct 9, 2017 at 8:23 PM, Padma Penumarthy <ppenumarthy@mapr.com>
wrote:

> which cloud service is this ?
> It is not able to read parquet metadata. Did you run refresh table
> metadata to
> generate parquet metadata ?
> Can you manually check if there is parquet metadata file
> (.drill.parquet_metadata)
> in the directory you used in the query i.e. `data25Goct6/websales` ?
>
> Thanks
> Padma
>
>
> On Oct 9, 2017, at 5:50 AM, PROJJWAL SAHA <proj.saha@gmail.com<mailto:pr
> oj.saha@gmail.com>> wrote:
>
> Hello all,
>
> I am getting the below exception when querying parquet data stored in
> storage cloud service.What does this exception point to ?
> The query on the same parquet files works when they are stored in
> alluxio.which means the data is fine.
> I am using drill 11.1
>
> Any help is appreciated !
>
> Regards,
> Projjwal
>
> 2017-10-09 08:11:10,221 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman]
> INFO  o.a.drill.exec.work.foreman.Foreman - Query text for query id
> 262498a1-4fc1-608e-a7bc-ab2c6ddc09c9: select count(*) from
> `data25Goct6/websales`
> 2017-10-09 08:11:38,117 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman]
> INFO  o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses()
> took 0 ms, numFiles: 1
> 2017-10-09 08:11:58,362 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman]
> INFO  o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses()
> took 0 ms, numFiles: 1
> 2017-10-09 08:15:28,459 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman]
> INFO  o.a.d.exec.store.parquet.Metadata - Took 105962 ms to get file
> statuses
> 2017-10-09 08:16:00,651 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman]
> ERROR o.a.d.exec.store.parquet.Metadata - Waited for 27187ms, but
> tasks for 'Fetch parquet metadata' are not complete. Total runnable
> size 29, parallelism 16.
> 2017-10-09 08:16:00,652 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman]
> INFO  o.a.d.exec.store.parquet.Metadata - User Error Occurred: Waited
> for 27187ms, but tasks for 'Fetch parquet metadata' are not complete.
> Total runnable size 29, parallelism
> 16.org.apache.drill.common.exceptions.UserException: RESOURCE ERROR:
> Waited for 27187ms, but tasks for 'Fetch parquet metadata' are not
> complete. Total runnable size 29, parallelism 16.
>
>
> [Error Id: d9b6ee72-2e81-49ae-846c-61a14931b7ab ]
> at org.apache.drill.common.exceptions.UserException$
> Builder.build(UserException.java:550)
> ~[drill-common-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.TimedRunnable.run(TimedRunnable.java:151)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.parquet.Metadata.getParquetFileMetadata_v3(
> Metadata.java:293)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.parquet.Metadata.getParquetTableMetadata(
> Metadata.java:270)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.parquet.Metadata.getParquetTableMetadata(
> Metadata.java:255)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.parquet.Metadata.getParquetTableMetadata(
> Metadata.java:117)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.parquet.ParquetGroupScan.init(
> ParquetGroupScan.java:730)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.parquet.ParquetGroupScan.<
> init>(ParquetGroupScan.java:226)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.parquet.ParquetGroupScan.<
> init>(ParquetGroupScan.java:186)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.parquet.ParquetFormatPlugin.getGroupScan(
> ParquetFormatPlugin.java:170)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.parquet.ParquetFormatPlugin.getGroupScan(
> ParquetFormatPlugin.java:66)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan(
> FileSystemPlugin.java:144)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan(
> AbstractStoragePlugin.java:100)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.planner.logical.DrillTable.
> getGroupScan(DrillTable.java:85)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.
> onMatch(DrillPushProjIntoScan.java:63)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.calcite.plan.volcano.VolcanoRuleCall.
> onMatch(VolcanoRuleCall.java:228)
> [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
> at org.apache.calcite.plan.volcano.VolcanoPlanner.
> findBestExp(VolcanoPlanner.java:811)
> [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
> at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:310)
> [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
> at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(
> DefaultSqlHandler.java:401)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(
> DefaultSqlHandler.java:343)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.
> convertToRawDrel(DefaultSqlHandler.java:242)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.
> convertToDrel(DefaultSqlHandler.java:292)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(
> DefaultSqlHandler.java:169)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.planner.sql.DrillSqlWorker.
> getQueryPlan(DrillSqlWorker.java:131)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(
> DrillSqlWorker.java:79)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1050)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:280)
> [drill-java-exec-1.11.0.jar:1.11.0]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message