beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anton Kedin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-3547) [SQL] Nested Query Generates Incompatible Trigger
Date Fri, 26 Jan 2018 23:07:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Anton Kedin updated BEAM-3547:
------------------------------
    Description: 
>From [https://stackoverflow.com/questions/48335383/nested-queries-in-beam-sql] :

 

SQL:
{code:java}
PCollection<BeamRecord> Query_Output = Query.apply(
            BeamSql.queryMulti("Select Orders.OrderID From Orders Where Orders.CustomerID
IN (Select Customers.CustomerID From Customers WHERE Customers.CustomerID = 2)"));{code}
 

Error:
{code:java}
org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner validateAndConvert
INFO: SQL:
SELECT `Orders`.`OrderID`
FROM `Orders` AS `Orders`
WHERE `Orders`.`CustomerID` IN (SELECT `Customers`.`CustomerID`
FROM `Customers` AS `Customers`
WHERE `Customers`.`CustomerID` = 2)
Jan 19, 2018 11:56:36 AM org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner
convertToBeamRel
INFO: SQLPlan>
LogicalProject(OrderID=[$0])
  LogicalJoin(condition=[=($1, $3)], joinType=[inner])
    LogicalTableScan(table=[[Orders]])
    LogicalAggregate(group=[{0}])
      LogicalProject(CustomerID=[$0])
        LogicalFilter(condition=[=($0, 2)])
          LogicalTableScan(table=[[Customers]])

Exception in thread "main" java.lang.IllegalStateException: java.lang.IllegalStateException:
Inputs to Flatten had incompatible triggers: DefaultTrigger, Repeatedly.forever(AfterWatermark.pastEndOfWindow())
    at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:165)
    at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:116)
    at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
    at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
    at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:160)
    at com.bitwise.cloud.ExampleOfJoins.main(ExampleOfJoins.java:91)
Caused by: java.lang.IllegalStateException: Inputs to Flatten had incompatible triggers: DefaultTrigger,
Repeatedly.forever(AfterWatermark.pastEndOfWindow())
    at org.apache.beam.sdk.transforms.Flatten$PCollections.expand(Flatten.java:123)
    at org.apache.beam.sdk.transforms.Flatten$PCollections.expand(Flatten.java:101)
    at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
    at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
    at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:182)
    at org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:124)
    at org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:74)
    at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
    at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
    at org.apache.beam.sdk.transforms.join.KeyedPCollectionTuple.apply(KeyedPCollectionTuple.java:107)
    at org.apache.beam.sdk.extensions.joinlibrary.Join.innerJoin(Join.java:59)
    at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.standardJoin(BeamJoinRel.java:217)
    at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.buildBeamPipeline(BeamJoinRel.java:161)
    at org.apache.beam.sdk.extensions.sql.impl.rel.BeamProjectRel.buildBeamPipeline(BeamProjectRel.java:68)
    at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:163)
    ... 5 more{code}

  was:
>From [https://beam.apache.org/documentation/programming-guide/#groupbykey-and-unbounded-pcollections] :

 

SQL:
{code:java}
PCollection<BeamRecord> Query_Output = Query.apply(
            BeamSql.queryMulti("Select Orders.OrderID From Orders Where Orders.CustomerID
IN (Select Customers.CustomerID From Customers WHERE Customers.CustomerID = 2)"));{code}
 

Error:
{code:java}
org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner validateAndConvert
INFO: SQL:
SELECT `Orders`.`OrderID`
FROM `Orders` AS `Orders`
WHERE `Orders`.`CustomerID` IN (SELECT `Customers`.`CustomerID`
FROM `Customers` AS `Customers`
WHERE `Customers`.`CustomerID` = 2)
Jan 19, 2018 11:56:36 AM org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner
convertToBeamRel
INFO: SQLPlan>
LogicalProject(OrderID=[$0])
  LogicalJoin(condition=[=($1, $3)], joinType=[inner])
    LogicalTableScan(table=[[Orders]])
    LogicalAggregate(group=[{0}])
      LogicalProject(CustomerID=[$0])
        LogicalFilter(condition=[=($0, 2)])
          LogicalTableScan(table=[[Customers]])

Exception in thread "main" java.lang.IllegalStateException: java.lang.IllegalStateException:
Inputs to Flatten had incompatible triggers: DefaultTrigger, Repeatedly.forever(AfterWatermark.pastEndOfWindow())
    at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:165)
    at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:116)
    at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
    at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
    at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:160)
    at com.bitwise.cloud.ExampleOfJoins.main(ExampleOfJoins.java:91)
Caused by: java.lang.IllegalStateException: Inputs to Flatten had incompatible triggers: DefaultTrigger,
Repeatedly.forever(AfterWatermark.pastEndOfWindow())
    at org.apache.beam.sdk.transforms.Flatten$PCollections.expand(Flatten.java:123)
    at org.apache.beam.sdk.transforms.Flatten$PCollections.expand(Flatten.java:101)
    at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
    at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
    at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:182)
    at org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:124)
    at org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:74)
    at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
    at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
    at org.apache.beam.sdk.transforms.join.KeyedPCollectionTuple.apply(KeyedPCollectionTuple.java:107)
    at org.apache.beam.sdk.extensions.joinlibrary.Join.innerJoin(Join.java:59)
    at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.standardJoin(BeamJoinRel.java:217)
    at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.buildBeamPipeline(BeamJoinRel.java:161)
    at org.apache.beam.sdk.extensions.sql.impl.rel.BeamProjectRel.buildBeamPipeline(BeamProjectRel.java:68)
    at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:163)
    ... 5 more{code}


> [SQL] Nested Query Generates Incompatible Trigger
> -------------------------------------------------
>
>                 Key: BEAM-3547
>                 URL: https://issues.apache.org/jira/browse/BEAM-3547
>             Project: Beam
>          Issue Type: Bug
>          Components: dsl-sql
>            Reporter: Anton Kedin
>            Priority: Major
>
> From [https://stackoverflow.com/questions/48335383/nested-queries-in-beam-sql] :
>  
> SQL:
> {code:java}
> PCollection<BeamRecord> Query_Output = Query.apply(
>             BeamSql.queryMulti("Select Orders.OrderID From Orders Where Orders.CustomerID
IN (Select Customers.CustomerID From Customers WHERE Customers.CustomerID = 2)"));{code}
>  
> Error:
> {code:java}
> org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner validateAndConvert
> INFO: SQL:
> SELECT `Orders`.`OrderID`
> FROM `Orders` AS `Orders`
> WHERE `Orders`.`CustomerID` IN (SELECT `Customers`.`CustomerID`
> FROM `Customers` AS `Customers`
> WHERE `Customers`.`CustomerID` = 2)
> Jan 19, 2018 11:56:36 AM org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner
convertToBeamRel
> INFO: SQLPlan>
> LogicalProject(OrderID=[$0])
>   LogicalJoin(condition=[=($1, $3)], joinType=[inner])
>     LogicalTableScan(table=[[Orders]])
>     LogicalAggregate(group=[{0}])
>       LogicalProject(CustomerID=[$0])
>         LogicalFilter(condition=[=($0, 2)])
>           LogicalTableScan(table=[[Customers]])
> Exception in thread "main" java.lang.IllegalStateException: java.lang.IllegalStateException:
Inputs to Flatten had incompatible triggers: DefaultTrigger, Repeatedly.forever(AfterWatermark.pastEndOfWindow())
>     at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:165)
>     at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:116)
>     at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
>     at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
>     at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:160)
>     at com.bitwise.cloud.ExampleOfJoins.main(ExampleOfJoins.java:91)
> Caused by: java.lang.IllegalStateException: Inputs to Flatten had incompatible triggers:
DefaultTrigger, Repeatedly.forever(AfterWatermark.pastEndOfWindow())
>     at org.apache.beam.sdk.transforms.Flatten$PCollections.expand(Flatten.java:123)
>     at org.apache.beam.sdk.transforms.Flatten$PCollections.expand(Flatten.java:101)
>     at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
>     at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
>     at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:182)
>     at org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:124)
>     at org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:74)
>     at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
>     at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
>     at org.apache.beam.sdk.transforms.join.KeyedPCollectionTuple.apply(KeyedPCollectionTuple.java:107)
>     at org.apache.beam.sdk.extensions.joinlibrary.Join.innerJoin(Join.java:59)
>     at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.standardJoin(BeamJoinRel.java:217)
>     at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.buildBeamPipeline(BeamJoinRel.java:161)
>     at org.apache.beam.sdk.extensions.sql.impl.rel.BeamProjectRel.buildBeamPipeline(BeamProjectRel.java:68)
>     at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:163)
>     ... 5 more{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message