flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1092) Allow operations, other than join and cogroup, on the solution set
Date Sat, 06 Sep 2014 15:55:28 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14124540#comment-14124540

Fabian Hueske commented on FLINK-1092:

I think the reason for this restriction is that the solution set is internally stored in a
distributed hash table.
In regular (non-iterative) processing, Flink uses hash tables only for join operations wich
access a hash table by probing it. Other operations like Map or Filter would require to read
the full hash table and my guess is that this has not been implemented because there was no
use case for that so far. Support for cogroup was probably added because it was required for
some algorithm.

[~StephanEwen] Is that correct?

> Allow operations, other than join and cogroup, on the solution set
> ------------------------------------------------------------------
>                 Key: FLINK-1092
>                 URL: https://issues.apache.org/jira/browse/FLINK-1092
>             Project: Flink
>          Issue Type: Improvement
>          Components: Iterations
>            Reporter: Vasia Kalavri
>            Priority: Minor
>              Labels: feature
> Currently, join and cogroup are the only operations allowed on the solution set.  We
should make the solution set usable for any type of operation, possibly using special functions.

This message was sent by Atlassian JIRA

View raw message