drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Khurram Faraaz <kfar...@mapr.com>
Subject Re: Drill “VALIDATION ERROR: A table or view with given name already exists in schema” for empty directory
Date Thu, 06 Dec 2018 01:22:58 GMT
Vitalii, this could be related to
https://issues.apache.org/jira/browse/DRILL-2775

Regards,
Khurram

On Wed, Dec 5, 2018 at 4:51 PM Vitalii Diravka <vitalii@apache.org> wrote:

> Hi Reed,
>
> It looks like a bug. Could you please create a jira ticket with an above
> description?
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_projects_DRILL_issues&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=H5JEl9vb-mBIjic10QAbDD2vkUUKAxjO6wZO322RtdI&m=WD8U6gAg6W1XWQoDvaWyzjdcvuEAdui_jCDb-8JOQhM&s=gNgq2I5icxbflP6fLgeNK5U2fF8N9vGBuokgEb03H6I&e=
>
> Kind regards
> Vitalii
>
>
> On Wed, Dec 5, 2018 at 6:57 PM Reed Villanueva <rvillanueva@ucera.org>
> wrote:
>
> > After upgrading drill on our cluster to drill-1.12.0-mapr, testing our
> > daily ETL scripts (which all use drill for converting parquet files to
> > tsv), a validation error ("*table or view with given name already
> exists*")
> > is always thrown when trying to run a `CREATE TABLE` statement on some
> > empty directories in a writable workspace.
> >
> >
> >     [Error Id: 6ea46737-8b6a-4887-a671-4bddbea02476 on
> > mapr002.ucera.local:31010]
> >     at
> >
> >
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:489)
> >     at
> >
> >
> org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(DrillCursor.java:561)
> >     :
> >     :
> >     :
> >     Caused by: org.apache.drill.common.exceptions.UserRemoteException:
> > VALIDATION ERROR: A table or view with given name
> > [/internal_etl/project/version-2/stages/storage/ACCOUNT/tsv] already
> exists
> > in schema [dfs.etl_internal]
> >
> >
> > After some brief debugging, I see that the directory in question under
> the
> > workspace (ie.
> /internal_etl/project/version-2/stages/storage/ACCOUNT/tsv)
> > *is in fact empty*, yet still throwing these errors.
> >
> > Looking for the error ID in the drillbit.log file in the associated node
> in
> > the error message above, we see
> >
> >     2018-12-04 10:13:25,285
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.a.drill.exec.work.foreman.Foreman - Query text for query id
> > 23f92019-db56-862f-e7b9-cd51b3e174ae: create table
> >
> >
> dfs.etl_internal.`/internal_etl/project/version-2/stages/storage/ACCOUNT/tsv`
> > as
> >     select <a bunch of fields>
> >     from
> >
> >
> dfs.etl_internal.`/internal_etl/project/version-2/stages/storage/ACCOUNT/parquet`
> >     2018-12-04 10:13:25,406
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses()
> took
> > 0 ms, numFiles: 1
> >     2018-12-04 10:13:25,408
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses()
> took
> > 0 ms, numFiles: 1
> >     2018-12-04 10:13:25,893
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses()
> took
> > 0 ms, numFiles: 1
> >     2018-12-04 10:13:25,894
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses()
> took
> > 0 ms, numFiles: 1
> >     2018-12-04 10:13:25,898
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses()
> took
> > 0 ms, numFiles: 1
> >     2018-12-04 10:13:25,898
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses()
> took
> > 0 ms, numFiles: 1
> >     2018-12-04 10:13:25,905
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.a.d.e.p.s.h.CreateTableHandler - User Error Occurred: A table or
> > view with given name
> > [/internal_etl/project/version-2/stages/storage/ACCOUNT/tsv] already
> exists
> > in schema [dfs.etl_internal]
> >     org.apache.drill.common.exceptions.UserException: VALIDATION ERROR: A
> > table or view with given name
> > [/internal_etl/project/version-2/stages/storage/ACCOUNT/tsv] already
> exists
> > in schema [dfs.etl_internal]
> >
> >
> >     [Error Id: 45177abc-7e9f-4678-959f-f9e0e38bc564 ]
> >     at
> >
> >
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:586)
> > ~[drill-common-1.12.0-mapr.jar:1.12.0-mapr]
> >     at
> >
> >
> org.apache.drill.exec.planner.sql.handlers.CreateTableHandler.checkTableCreationPossibility(CreateTableHandler.java:326)
> > [drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]
> >     at
> >
> >
> org.apache.drill.exec.planner.sql.handlers.CreateTableHandler.getPlan(CreateTableHandler.java:90)
> > [drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]
> >     at
> >
> >
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:131)
> > [drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]
> >     at
> >
> >
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:79)
> > [drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]
> >     at org.apache.drill.exec.work
> .foreman.Foreman.runSQL(Foreman.java:567)
> > [drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]
> >     at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
> > [drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]
> >     at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > [na:1.8.0_151]
> >     at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > [na:1.8.0_151]
> >     at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
> >     2018-12-04 10:13:25,924
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.apache.drill.exec.work.WorkManager - Waiting for 0 queries to
> > complete before shutting down
> >     2018-12-04 10:13:25,924
> [23f92019-db56-862f-e7b9-cd51b3e174ae:foreman]
> > INFO  o.apache.drill.exec.work.WorkManager - Waiting for 0 running
> > fragments to complete before shutting down
> >
> > This error occurs even when using `DROP TABLE [IF EXISTS]
> > <workspace>.<table path name>` before the `CREATE TABLE` statement.
> > Furthermore, the configurations for the dfs workspace itself does not
> > appear to be changed from before upgrading to drill-1.12, see below:
> >
> >     :
> >     :
> >     "workspaces": {
> >     "root": {
> >     "location": "/",
> >     "writable": false,
> >     "defaultInputFormat": null,
> >     "allowAccessOutsideWorkspace": false
> >     },
> >     "tmp": {
> >     "location": "/tmp",
> >     "writable": true,
> >     "defaultInputFormat": null,
> >     "allowAccessOutsideWorkspace": false
> >     },
> >     "etl_internal": {
> >     "location": "/etl/internal",
> >     "writable": true,
> >     "defaultInputFormat": null,
> >     "allowAccessOutsideWorkspace": false
> >     }
> >     },
> >     :
> >     :
> >
> > Note that the full process in question is intended to `mv` the directory
> > contents every day and `CREATE TABLE` with new data from current day (in
> > case that makes a difference) and this process had been working fine when
> > we were using drill-1.11.
> >
> > If anyone with more experience using drill knows what could be happening
> > here, any opinions or advice would be appreciated.
> >
> > --
> > This electronic message is intended only for the named
> > recipient, and may
> > contain information that is confidential or
> > privileged. If you are not the
> > intended recipient, you are
> > hereby notified that any disclosure, copying,
> > distribution or
> > use of the contents of this message is strictly
> > prohibited. If
> > you have received this message in error or are not the
> > named
> > recipient, please notify us immediately by contacting the
> > sender at
> > the electronic mail address noted above, and delete
> > and destroy all copies
> > of this message. Thank you.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message