drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Givre <cgi...@gmail.com>
Subject Re: HDFS file is listable but not queryable (object not found)
Date Thu, 23 Jul 2020 15:30:23 GMT
Hi Clark, 
That's strange.  My initial thought is that this could be a permission issue.  However, it
might also be that Drill isn't finding the file for some reason. 

Could you try:

SELECT * 
FROM hdfs.`<full hdfs path to file>`

Best,
--- C


> On Jul 23, 2020, at 11:23 AM, Updike, Clark <Clark.Updike@jhuapl.edu> wrote:
> 
> This is in 1.17.  I can use SHOW FILES to list the file I'm targeting, but I cannot query
it:
> 
> apache drill> show files in hdfs.root.`/tmp/employee.json`;
> +---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
> |     name      | isDirectory | isFile | length |  owner   |   group    | permissions
|       accessTime        |    modificationTime     |
> +---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
> | employee.json | false       | true   | 474630 | me       | supergroup | rw-r--r-- 
 | 2020-07-23 10:53:15.055 | 2020-07-23 10:53:15.387 |
> +---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
> 1 row selected (3.039 seconds)
> 
> 
> apache drill> select * from hdfs.root.`/tmp/employee.json`;
> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 18: Object '/tmp/employee.json'
not found within 'hdfs.root'
> [Error Id: 3b833622-4fac-4ecc-becd-118291cd8560 ] (state=,code=0)
> 
> The storage plugin uses the standard json config:
> 
>    "json": {
>      "type": "json",
>      "extensions": [
>        "json"
>      ]
>    },
> 
> I can't see any problems on the HDFS side.  Full stack trace is below.
> 
> Any ideas what could be causing this behavior?
> 
> Thanks, Clark
> 
> 
> 
> FULL STACKTRACE:
> 
> apache drill> select * from hdfs.root.`/tmp/employee.json`;
> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 18: Object '/tmp/employee.json'
not found within 'hdfs.root'
> 
> 
> [Error Id: 69c8ffc0-4933-4008-a786-85ad623578ea ]
> 
>  (org.apache.calcite.runtime.CalciteContextException) From line 1, column 15 to line
1, column 18: Object '/tmp/employee.json' not found within 'hdfs.root'
>    sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
>    sun.reflect.NativeConstructorAccessorImpl.newInstance():62
>    sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
>    java.lang.reflect.Constructor.newInstance():423
>    org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
>    org.apache.calcite.sql.SqlUtil.newContextException():824
>    org.apache.calcite.sql.SqlUtil.newContextException():809
>    org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4805
>    org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127
>    org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177
>    org.apache.calcite.sql.validate.AbstractNamespace.validate():84
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3109
>    org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3091
>    org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3363
>    org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
>    org.apache.calcite.sql.validate.AbstractNamespace.validate():84
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
>    org.apache.calcite.sql.SqlSelect.validate():216
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():930
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validate():637
>    org.apache.drill.exec.planner.sql.SqlConverter.validate():218
>    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():665
>    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():199
>    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
>    org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():282
>    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan():162
>    org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():127
>    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():92
>    org.apache.drill.exec.work.foreman.Foreman.runSQL():590
>    org.apache.drill.exec.work.foreman.Foreman.run():275
>    java.util.concurrent.ThreadPoolExecutor.runWorker():1142
>    java.util.concurrent.ThreadPoolExecutor$Worker.run():617
>    java.lang.Thread.run():745
>  Caused By (org.apache.calcite.sql.validate.SqlValidatorException) Object '/tmp/employee.json'
not found within 'hdfs.root'
>    sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
>    sun.reflect.NativeConstructorAccessorImpl.newInstance():62
>    sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
>    java.lang.reflect.Constructor.newInstance():423
>    org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
>    org.apache.calcite.runtime.Resources$ExInst.ex():572
>    org.apache.calcite.sql.SqlUtil.newContextException():824
>    org.apache.calcite.sql.SqlUtil.newContextException():809
>    org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4805
>    org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127
>    org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177
>    org.apache.calcite.sql.validate.AbstractNamespace.validate():84
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3109
>    org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3091
>    org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3363
>    org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
>    org.apache.calcite.sql.validate.AbstractNamespace.validate():84
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
>    org.apache.calcite.sql.SqlSelect.validate():216
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():930
>    org.apache.calcite.sql.validate.SqlValidatorImpl.validate():637
>    org.apache.drill.exec.planner.sql.SqlConverter.validate():218
>    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():665
>    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():199
>    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
>    org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():282
>    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan():162
>    org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():127
>    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():92
>    org.apache.drill.exec.work.foreman.Foreman.runSQL():590
>    org.apache.drill.exec.work.foreman.Foreman.run():275
>    java.util.concurrent.ThreadPoolExecutor.runWorker():1142
>    java.util.concurrent.ThreadPoolExecutor$Worker.run():617
>    java.lang.Thread.run():745 (state=,code=0)


Mime
View raw message