flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3943) Add support for EXCEPT (set minus)
Date Thu, 30 Jun 2016 03:40:12 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356385#comment-15356385
] 

ASF GitHub Bot commented on FLINK-3943:
---------------------------------------

Github user wuchong commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2169#discussion_r69065352
  
    --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/plan/logical/operators.scala
---
    @@ -236,6 +236,32 @@ case class Aggregate(
       }
     }
     
    +case class SetMinus(left: LogicalNode, right: LogicalNode, all: Boolean) extends BinaryNode
{
    +  override def output: Seq[Attribute] = left.output
    +
    +  override protected[logical] def construct(relBuilder: RelBuilder): RelBuilder = {
    +    left.construct(relBuilder)
    +    right.construct(relBuilder)
    +    relBuilder.minus(all)
    +  }
    +
    +  override def validate(tableEnv: TableEnvironment): LogicalNode = {
    +    val resolvedMinus = super.validate(tableEnv).asInstanceOf[SetMinus]
    +    if (left.output.length != right.output.length) {
    +      failValidation(s"Set minus two table of different column sizes:" +
    +        s" ${left.output.size} and ${right.output.size}")
    +    }
    +    val sameSchema = left.output.zip(right.output).forall { case (l, r) =>
    +      l.resultType == r.resultType && l.name == r.name }
    --- End diff --
    
    Yes, I refer to the last case. I agree with @fhueske 's opinion, we can remove the check
of field names in `EXCEPT` and `INTERSECT` now, and remove the restriction in `UNION` in the
future. 


> Add support for EXCEPT (set minus)
> ----------------------------------
>
>                 Key: FLINK-3943
>                 URL: https://issues.apache.org/jira/browse/FLINK-3943
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>    Affects Versions: 1.1.0
>            Reporter: Fabian Hueske
>            Assignee: Ivan Mushketyk
>            Priority: Minor
>
> Currently, the Table API and SQL do not support EXCEPT.
> EXCEPT can be executed as a coGroup on all fields that forwards records of the first
input if the second input is empty.
> In order to add support for EXCEPT to the Table API and SQL we need to:
> - Implement a {{DataSetMinus}} class that translates an EXCEPT into a DataSet API program
using a coGroup on all fields.
> - Implement a {{DataSetMinusRule}} that translates a Calcite {{LogicalMinus}} into a
{{DataSetMinus}}.
> - Extend the Table API (and validation phase) to provide an except() method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message