drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Updike, Clark" <Clark.Upd...@jhuapl.edu>
Subject Re: RE: HDFS file is listable but not queryable (object not found)
Date Thu, 23 Jul 2020 18:17:57 GMT
No change unfortunately:

apache drill> select * from hdfs.`root`.`/tmp/employee.json`;
Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 18: Object '/tmp/employee.json'
not found within 'hdfs.root'

´╗┐On 7/23/20, 2:11 PM, "Paul Rogers" <par0328@gmail.com> wrote:

    Hi Clark,
    
    Try using `hdfs`.`root` rather than `hdfs.root`. Calcite wants to walk down
    `hdfs` then `root`. There is no workspace called `hdfs.root`.
    
    Thanks,
    
    - Paul
    
    On Thu, Jul 23, 2020 at 8:58 AM Updike, Clark <Clark.Updike@jhuapl.edu>
    wrote:
    
    > Oops, sorry.  No luck there either unfortunately:
    >
    > apache drill> SELECT * FROM hdfs.`/tmp/employee.json`;
    > Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 18:
    > Object '/tmp/employee.json' not found within 'hdfs'
    >
    >
    > On 7/23/20, 11:52 AM, "Charles Givre" <cgivre@gmail.com> wrote:
    >
    >     Oh.. I meant:
    >
    >     SELECT *
    >     FROM hdfs.`/tmp/employee.json`
    >
    >     > On Jul 23, 2020, at 11:41 AM, Updike, Clark <Clark.Updike@jhuapl.edu>
    > wrote:
    >     >
    >     > No change unfortunately...
    >     >
    >     > $ hdfs dfs -ls hdfs://nn01:8020/tmp/employee.json
    >     > -rw-r--r--   2 me supergroup     474630 2020-07-23 10:53
    > hdfs://nn01:8020/tmp/employee.json
    >     >
    >     > apache drill> select * from
    > hdfs.root.`hdfs://nn01:8020/tmp/employee.json`;
    >     > Error: VALIDATION ERROR: From line 1, column 15 to line 1, column
    > 18: Object 'hdfs://nn01:8020/tmp/employee.json' not found within 'hdfs.root'
    >     >
    >     >
    >     > On 7/23/20, 11:30 AM, "Charles Givre" <cgivre@gmail.com> wrote:
    >     >
    >     >    Hi Clark,
    >     >    That's strange.  My initial thought is that this could be a
    > permission issue.  However, it might also be that Drill isn't finding the
    > file for some reason.
    >     >
    >     >    Could you try:
    >     >
    >     >    SELECT *
    >     >    FROM hdfs.`<full hdfs path to file>`
    >     >
    >     >    Best,
    >     >    --- C
    >     >
    >     >
    >     >> On Jul 23, 2020, at 11:23 AM, Updike, Clark <
    > Clark.Updike@jhuapl.edu> wrote:
    >     >>
    >     >> This is in 1.17.  I can use SHOW FILES to list the file I'm
    > targeting, but I cannot query it:
    >     >>
    >     >> apache drill> show files in hdfs.root.`/tmp/employee.json`;
    >     >>
    > +---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
    >     >> |     name      | isDirectory | isFile | length |  owner   |
    >  group    | permissions |       accessTime        |    modificationTime
    >  |
    >     >>
    > +---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
    >     >> | employee.json | false       | true   | 474630 | me       |
    > supergroup | rw-r--r--   | 2020-07-23 10:53:15.055 | 2020-07-23
    > 10:53:15.387 |
    >     >>
    > +---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
    >     >> 1 row selected (3.039 seconds)
    >     >>
    >     >>
    >     >> apache drill> select * from hdfs.root.`/tmp/employee.json`;
    >     >> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column
    > 18: Object '/tmp/employee.json' not found within 'hdfs.root'
    >     >> [Error Id: 3b833622-4fac-4ecc-becd-118291cd8560 ] (state=,code=0)
    >     >>
    >     >> The storage plugin uses the standard json config:
    >     >>
    >     >>   "json": {
    >     >>     "type": "json",
    >     >>     "extensions": [
    >     >>       "json"
    >     >>     ]
    >     >>   },
    >     >>
    >     >> I can't see any problems on the HDFS side.  Full stack trace is
    > below.
    >     >>
    >     >> Any ideas what could be causing this behavior?
    >     >>
    >     >> Thanks, Clark
    >     >>
    >     >>
    >     >>
    >     >> FULL STACKTRACE:
    >     >>
    >     >> apache drill> select * from hdfs.root.`/tmp/employee.json`;
    >     >> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column
    > 18: Object '/tmp/employee.json' not found within 'hdfs.root'
    >     >>
    >     >>
    >     >> [Error Id: 69c8ffc0-4933-4008-a786-85ad623578ea ]
    >     >>
    >     >> (org.apache.calcite.runtime.CalciteContextException) From line 1,
    > column 15 to line 1, column 18: Object '/tmp/employee.json' not found
    > within 'hdfs.root'
    >     >>   sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
    >     >>   sun.reflect.NativeConstructorAccessorImpl.newInstance():62
    >     >>   sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
    >     >>   java.lang.reflect.Constructor.newInstance():423
    >     >>   org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
    >     >>   org.apache.calcite.sql.SqlUtil.newContextException():824
    >     >>   org.apache.calcite.sql.SqlUtil.newContextException():809
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4805
    >     >>
    >  org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127
    >     >>
    >  org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177
    >     >>   org.apache.calcite.sql.validate.AbstractNamespace.validate():84
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3109
    >     >>
    >  org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3091
    >     >>
    >  org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3363
    >     >>   org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
    >     >>   org.apache.calcite.sql.validate.AbstractNamespace.validate():84
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
    >     >>   org.apache.calcite.sql.SqlSelect.validate():216
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():930
    >     >>   org.apache.calcite.sql.validate.SqlValidatorImpl.validate():637
    >     >>   org.apache.drill.exec.planner.sql.SqlConverter.validate():218
    >     >>
    >  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():665
    >     >>
    >  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():199
    >     >>
    >  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
    >     >>
    >  org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():282
    >     >>
    >  org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan():162
    >     >>   org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():127
    >     >>   org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():92
    >     >>   org.apache.drill.exec.work.foreman.Foreman.runSQL():590
    >     >>   org.apache.drill.exec.work.foreman.Foreman.run():275
    >     >>   java.util.concurrent.ThreadPoolExecutor.runWorker():1142
    >     >>   java.util.concurrent.ThreadPoolExecutor$Worker.run():617
    >     >>   java.lang.Thread.run():745
    >     >> Caused By (org.apache.calcite.sql.validate.SqlValidatorException)
    > Object '/tmp/employee.json' not found within 'hdfs.root'
    >     >>   sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
    >     >>   sun.reflect.NativeConstructorAccessorImpl.newInstance():62
    >     >>   sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
    >     >>   java.lang.reflect.Constructor.newInstance():423
    >     >>   org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
    >     >>   org.apache.calcite.runtime.Resources$ExInst.ex():572
    >     >>   org.apache.calcite.sql.SqlUtil.newContextException():824
    >     >>   org.apache.calcite.sql.SqlUtil.newContextException():809
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4805
    >     >>
    >  org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127
    >     >>
    >  org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177
    >     >>   org.apache.calcite.sql.validate.AbstractNamespace.validate():84
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3109
    >     >>
    >  org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3091
    >     >>
    >  org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3363
    >     >>   org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
    >     >>   org.apache.calcite.sql.validate.AbstractNamespace.validate():84
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
    >     >>   org.apache.calcite.sql.SqlSelect.validate():216
    >     >>
    >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():930
    >     >>   org.apache.calcite.sql.validate.SqlValidatorImpl.validate():637
    >     >>   org.apache.drill.exec.planner.sql.SqlConverter.validate():218
    >     >>
    >  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():665
    >     >>
    >  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():199
    >     >>
    >  org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
    >     >>
    >  org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():282
    >     >>
    >  org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan():162
    >     >>   org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():127
    >     >>   org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():92
    >     >>   org.apache.drill.exec.work.foreman.Foreman.runSQL():590
    >     >>   org.apache.drill.exec.work.foreman.Foreman.run():275
    >     >>   java.util.concurrent.ThreadPoolExecutor.runWorker():1142
    >     >>   java.util.concurrent.ThreadPoolExecutor$Worker.run():617
    >     >>   java.lang.Thread.run():745 (state=,code=0)
    >     >
    >     >
    >     >
    >
    >
    >
    >
    

Mime
View raw message