drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Rogers <par0...@gmail.com>
Subject Re: Re: HDFS file is listable but not queryable (object not found)
Date Thu, 23 Jul 2020 18:10:42 GMT
Hi Clark,

Try using `hdfs`.`root` rather than `hdfs.root`. Calcite wants to walk down
`hdfs` then `root`. There is no workspace called `hdfs.root`.

Thanks,

- Paul

On Thu, Jul 23, 2020 at 8:58 AM Updike, Clark <Clark.Updike@jhuapl.edu>
wrote:

> Oops, sorry.  No luck there either unfortunately:
>
> apache drill> SELECT * FROM hdfs.`/tmp/employee.json`;
> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 18:
> Object '/tmp/employee.json' not found within 'hdfs'
>
>
> ´╗┐On 7/23/20, 11:52 AM, "Charles Givre" <cgivre@gmail.com> wrote:
>
>     Oh.. I meant:
>
>     SELECT *
>     FROM hdfs.`/tmp/employee.json`
>
>     > On Jul 23, 2020, at 11:41 AM, Updike, Clark <Clark.Updike@jhuapl.edu>
> wrote:
>     >
>     > No change unfortunately...
>     >
>     > $ hdfs dfs -ls hdfs://nn01:8020/tmp/employee.json
>     > -rw-r--r--   2 me supergroup     474630 2020-07-23 10:53
> hdfs://nn01:8020/tmp/employee.json
>     >
>     > apache drill> select * from
> hdfs.root.`hdfs://nn01:8020/tmp/employee.json`;
>     > Error: VALIDATION ERROR: From line 1, column 15 to line 1, column
> 18: Object 'hdfs://nn01:8020/tmp/employee.json' not found within 'hdfs.root'
>     >
>     >
>     > On 7/23/20, 11:30 AM, "Charles Givre" <cgivre@gmail.com> wrote:
>     >
>     >    Hi Clark,
>     >    That's strange.  My initial thought is that this could be a
> permission issue.  However, it might also be that Drill isn't finding the
> file for some reason.
>     >
>     >    Could you try:
>     >
>     >    SELECT *
>     >    FROM hdfs.`<full hdfs path to file>`
>     >
>     >    Best,
>     >    --- C
>     >
>     >
>     >> On Jul 23, 2020, at 11:23 AM, Updike, Clark <
> Clark.Updike@jhuapl.edu> wrote:
>     >>
>     >> This is in 1.17.  I can use SHOW FILES to list the file I'm
> targeting, but I cannot query it:
>     >>
>     >> apache drill> show files in hdfs.root.`/tmp/employee.json`;
>     >>
> +---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
>     >> |     name      | isDirectory | isFile | length |  owner   |
>  group    | permissions |       accessTime        |    modificationTime
>  |
>     >>
> +---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
>     >> | employee.json | false       | true   | 474630 | me       |
> supergroup | rw-r--r--   | 2020-07-23 10:53:15.055 | 2020-07-23
> 10:53:15.387 |
>     >>
> +---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
>     >> 1 row selected (3.039 seconds)
>     >>
>     >>
>     >> apache drill> select * from hdfs.root.`/tmp/employee.json`;
>     >> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column
> 18: Object '/tmp/employee.json' not found within 'hdfs.root'
>     >> [Error Id: 3b833622-4fac-4ecc-becd-118291cd8560 ] (state=,code=0)
>     >>
>     >> The storage plugin uses the standard json config:
>     >>
>     >>   "json": {
>     >>     "type": "json",
>     >>     "extensions": [
>     >>       "json"
>     >>     ]
>     >>   },
>     >>
>     >> I can't see any problems on the HDFS side.  Full stack trace is
> below.
>     >>
>     >> Any ideas what could be causing this behavior?
>     >>
>     >> Thanks, Clark
>     >>
>     >>
>     >>
>     >> FULL STACKTRACE:
>     >>
>     >> apache drill> select * from hdfs.root.`/tmp/employee.json`;
>     >> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column
> 18: Object '/tmp/employee.json' not found within 'hdfs.root'
>     >>
>     >>
>     >> [Error Id: 69c8ffc0-4933-4008-a786-85ad623578ea ]
>     >>
>     >> (org.apache.calcite.runtime.CalciteContextException) From line 1,
> column 15 to line 1, column 18: Object '/tmp/employee.json' not found
> within 'hdfs.root'
>     >>   sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
>     >>   sun.reflect.NativeConstructorAccessorImpl.newInstance():62
>     >>   sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
>     >>   java.lang.reflect.Constructor.newInstance():423
>     >>   org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
>     >>   org.apache.calcite.sql.SqlUtil.newContextException():824
>     >>   org.apache.calcite.sql.SqlUtil.newContextException():809
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4805
>     >>
>  org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127
>     >>
>  org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177
>     >>   org.apache.calcite.sql.validate.AbstractNamespace.validate():84
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3109
>     >>
>  org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3091
>     >>
>  org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3363
>     >>   org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
>     >>   org.apache.calcite.sql.validate.AbstractNamespace.validate():84
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
>     >>   org.apache.calcite.sql.SqlSelect.validate():216
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():930
>     >>   org.apache.calcite.sql.validate.SqlValidatorImpl.validate():637
>     >>   org.apache.drill.exec.planner.sql.SqlConverter.validate():218
>     >>
>  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():665
>     >>
>  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():199
>     >>
>  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
>     >>
>  org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():282
>     >>
>  org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan():162
>     >>   org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():127
>     >>   org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():92
>     >>   org.apache.drill.exec.work.foreman.Foreman.runSQL():590
>     >>   org.apache.drill.exec.work.foreman.Foreman.run():275
>     >>   java.util.concurrent.ThreadPoolExecutor.runWorker():1142
>     >>   java.util.concurrent.ThreadPoolExecutor$Worker.run():617
>     >>   java.lang.Thread.run():745
>     >> Caused By (org.apache.calcite.sql.validate.SqlValidatorException)
> Object '/tmp/employee.json' not found within 'hdfs.root'
>     >>   sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
>     >>   sun.reflect.NativeConstructorAccessorImpl.newInstance():62
>     >>   sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
>     >>   java.lang.reflect.Constructor.newInstance():423
>     >>   org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
>     >>   org.apache.calcite.runtime.Resources$ExInst.ex():572
>     >>   org.apache.calcite.sql.SqlUtil.newContextException():824
>     >>   org.apache.calcite.sql.SqlUtil.newContextException():809
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4805
>     >>
>  org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127
>     >>
>  org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177
>     >>   org.apache.calcite.sql.validate.AbstractNamespace.validate():84
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3109
>     >>
>  org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3091
>     >>
>  org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3363
>     >>   org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
>     >>   org.apache.calcite.sql.validate.AbstractNamespace.validate():84
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
>     >>   org.apache.calcite.sql.SqlSelect.validate():216
>     >>
>  org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():930
>     >>   org.apache.calcite.sql.validate.SqlValidatorImpl.validate():637
>     >>   org.apache.drill.exec.planner.sql.SqlConverter.validate():218
>     >>
>  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():665
>     >>
>  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():199
>     >>
>  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
>     >>
>  org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():282
>     >>
>  org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan():162
>     >>   org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():127
>     >>   org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():92
>     >>   org.apache.drill.exec.work.foreman.Foreman.runSQL():590
>     >>   org.apache.drill.exec.work.foreman.Foreman.run():275
>     >>   java.util.concurrent.ThreadPoolExecutor.runWorker():1142
>     >>   java.util.concurrent.ThreadPoolExecutor$Worker.run():617
>     >>   java.lang.Thread.run():745 (state=,code=0)
>     >
>     >
>     >
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message