drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muhammad Gelbana <m.gelb...@gmail.com>
Subject Re: FindHardDistributionScans throws a NPE while visiting a TableScan
Date Sat, 24 Jun 2017 14:12:46 GMT
With pleasure. I can't successfully run all test cases
<https://issues.apache.org/jira/browse/DRILL-5606> ATM. When I overcome
that, I'll push the fix.

*---------------------*
*Muhammad Gelbana*
http://www.linkedin.com/in/mgelbana

On Wed, Jun 21, 2017 at 1:47 PM, Khurram Faraaz <kfaraaz@mapr.com> wrote:

> Muhammad, please create a pull request and someone will review you code,
> ensure that existing unit tests don't fail due to your changes.
>
>
> Thanks,
>
> Khurram
>
> ________________________________
> From: Muhammad Gelbana <m.gelbana@gmail.com>
> Sent: Wednesday, June 21, 2017 4:11:41 PM
> To: dev@drill.apache.org
> Subject: Re: FindHardDistributionScans throws a NPE while visiting a
> TableScan
>
> This has been bugging me for sometime, and I've only solved it after
> starting this thread !
>
> I solved this by overriding the
> *org.apache.calcite.rel.AbstractRelNode.accept(RelShuttle)* method for the
> relational node(s) containing *JdbcTableScan* to avoid this.
>
> @Override
> >     public RelNode accept(RelShuttle shuttle) {
> >
> > if(shuttle.getClass().getName().equals("org.apache.drill.
> exec.planner.sql.handlers.FindHardDistributionScans")){
> >         return this;
> >     }
> >     return super.accept(shuttle);
> > }
>
>
> If someone finds this introducing another bug, please tell me about it.
>
>
> *---------------------*
> *Muhammad Gelbana*
> http://www.linkedin.com/in/mgelbana
>
> On Tue, Jun 20, 2017 at 2:13 AM, Jinfeng Ni <jni@apache.org> wrote:
>
> > unwrap() essentially is doing a cast.  If it returns null for
> > unwrap(DrillTranslatableTable.class) or unwrap(DrillTable.class), it
> means
> > the table associate with this TableScan does not implement either
> > interface. My suspicion is  JDBC storage plugin returns JdbcTable [1],
> > unlikely other storage plugin which returns an instance implementing
> > DrillTable.
> >
> > This seems to indicate FindHardDistributionScans could not be used to
> > non-DrillTable. I'm not sure if that's the intention of that code,
> though.
> >
> > 1.
> > https://github.com/apache/calcite/blob/master/core/src/
> > main/java/org/apache/calcite/adapter/jdbc/JdbcSchema.java#L233-L234
> >
> > On Mon, Jun 19, 2017 at 2:19 PM, Muhammad Gelbana <m.gelbana@gmail.com>
> > wrote:
> >
> > > Everyone,
> > >
> > > I made a copy of the Jdbc plugin and made modifications to it by
> adding a
> > > few rules. None of the modification I made or the rules I wrote should
> > have
> > > anything extra to do with handling the following SQL query
> > >
> > > SELECT * FROM incorta.SALES.SALES SALES WHERE 1 = 2 LIMIT 1
> > >
> > >
> > > I know the query is useless, but I need to to know how to fix the
> > following
> > > error thrown while handling this query. This is the final query plan:
> > >
> > > DEBUG o.a.d.e.p.s.h.DefaultSqlHandler - HEP_BOTTOM_UP:Convert SUM to
> > $SUM0
> > > > (0ms):
> > > > DrillLimitRel(*fetch=[1]*): rowcount = 1.0, cumulative cost = {201.0
> > > > rows, 205.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2653
> > > >   DrillLimitRel(*offset=[0], fetch=[0]*): rowcount = 1.0, cumulative
> > cost
> > > > = {200.0 rows, 201.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2651
> > > >     GelbanaJdbcDrel: rowcount = 100.0, cumulative cost = {200.0 rows,
> > > > 201.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2649
> > > >       JdbcTableScan(table=[[gelbana, SALES, SALES]]): rowcount =
> 100.0,
> > > > cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0
> > > memory},
> > > > id = 2572
> > >
> > >
> > > This is the throw error stacktrace
> > >
> > > [Error Id: 83ea094a-db24-4d6d-bf0d-271db26db933 on 640fb7ebbd1a:31010]
> > > at
> > > org.apache.drill.common.exceptions.UserException$
> > > Builder.build(UserException.java:543)
> > > ~[drill-common-1.9.0.jar:1.9.0]
> > > at
> > > org.apache.drill.exec.work.foreman.Foreman$ForemanResult.
> > > close(Foreman.java:825)
> > > [drill-java-exec-1.9.0.jar:1.9.0]
> > > at org.apache.drill.exec.work.foreman.Foreman.moveToState(
> > > Foreman.java:935)
> > > [drill-java-exec-1.9.0.jar:1.9.0]
> > > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281)
> > > [drill-java-exec-1.9.0.jar:1.9.0]
> > > at
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > ThreadPoolExecutor.java:1142)
> > > [na:1.8.0_131]
> > > at
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > ThreadPoolExecutor.java:617)
> > > [na:1.8.0_131]
> > > at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> > > Caused by: org.apache.drill.exec.work.foreman.ForemanException:
> > Unexpected
> > > exception during fragment initialization: null
> > > ... 4 common frames omitted
> > > *Caused by: java.lang.NullPointerException: null*
> > > at
> > > org.apache.drill.exec.planner.sql.handlers.FindHardDistributionScans.
> > > visit(FindHardDistributionScans.java:55)
> > > ~[drill-java-exec-1.9.0.jar:1.9.0]
> > > at org.apache.calcite.rel.core.TableScan.accept(TableScan.java:166)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.RelShuttleImpl.visitChild(
> > > RelShuttleImpl.java:53)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at
> > > org.apache.calcite.rel.RelShuttleImpl.visitChildren(
> > > RelShuttleImpl.java:68)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.RelShuttleImpl.visit(
> RelShuttleImpl.java:126)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.AbstractRelNode.accept(
> > AbstractRelNode.java:256)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.RelShuttleImpl.visitChild(
> > > RelShuttleImpl.java:53)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at
> > > org.apache.calcite.rel.RelShuttleImpl.visitChildren(
> > > RelShuttleImpl.java:68)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.RelShuttleImpl.visit(
> RelShuttleImpl.java:126)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.AbstractRelNode.accept(
> > AbstractRelNode.java:256)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.RelShuttleImpl.visitChild(
> > > RelShuttleImpl.java:53)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at
> > > org.apache.calcite.rel.RelShuttleImpl.visitChildren(
> > > RelShuttleImpl.java:68)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.RelShuttleImpl.visit(
> RelShuttleImpl.java:126)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.AbstractRelNode.accept(
> > AbstractRelNode.java:256)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at
> > > org.apache.drill.exec.planner.sql.handlers.FindHardDistributionScans.
> > > canForceSingleMode(FindHardDistributionScans.java:45)
> > > ~[drill-java-exec-1.9.0.jar:1.9.0]
> > > at
> > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.
> > > convertToDrel(DefaultSqlHandler.java:262)
> > > ~[drill-java-exec-1.9.0.jar:1.9.0]
> > > at
> > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.
> > > convertToDrel(DefaultSqlHandler.java:290)
> > > ~[drill-java-exec-1.9.0.jar:1.9.0]
> > > at
> > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(
> > > DefaultSqlHandler.java:168)
> > > ~[drill-java-exec-1.9.0.jar:1.9.0]
> > > at
> > > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(
> > > DrillSqlWorker.java:123)
> > > ~[drill-java-exec-1.9.0.jar:1.9.0]
> > > at
> > > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(
> > > DrillSqlWorker.java:97)
> > > ~[drill-java-exec-1.9.0.jar:1.9.0]
> > > at org.apache.drill.exec.work.foreman.Foreman.runSQL(
> Foreman.java:1008)
> > > [drill-java-exec-1.9.0.jar:1.9.0]
> > > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
> > > [drill-java-exec-1.9.0.jar:1.9.0]
> > > ... 3 common frames omitted
> > >
> > > Whic is because this statement
> > >
> > > unwrap =
> > > > scan.getTable().unwrap(DrillTranslatableTable.class).
> getDrillTable();
> > >
> > >
> > > In *FindHardDistributionScans.java:55* evaluates
> > > *scan.getTable().unwrap(DrillTranslatableTable.class)* to null
> > >
> > > Would someone please explain to me what is Drill trying to do and what
> > did
> > > I do wrong ?
> > >
> > > *---------------------*
> > > *Muhammad Gelbana*
> > > http://www.linkedin.com/in/mgelbana
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message