flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Shoshin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3133) Introduce collect()/count()/print() methods in DataStream API
Date Fri, 18 Nov 2016 08:43:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676175#comment-15676175
] 

Alexander Shoshin commented on FLINK-3133:
------------------------------------------

Hi )

Thanks, Maximilian. I think that now I understand this issue. One more question: when we call
{{jobClient.sampleStream(..)}}, should we immediatly recieve the result of the last 5 seconds
(then what if the job is just started and we have not yet have any results?) or we should
block execution for 5 seconds till the result will be accumulated?

And I see that {{JobClient}} is going to be implemented in FLINK-4272. Should we wait till
it will be merged?

> Introduce collect()/count()/print() methods in DataStream API
> -------------------------------------------------------------
>
>                 Key: FLINK-3133
>                 URL: https://issues.apache.org/jira/browse/FLINK-3133
>             Project: Flink
>          Issue Type: Improvement
>          Components: DataStream API, Streaming
>    Affects Versions: 0.10.0, 1.0.0, 0.10.1
>            Reporter: Maximilian Michels
>            Assignee: Alexander Shoshin
>             Fix For: 1.0.0
>
>
> The DataSet API's methods {{collect()}}, {{count()}}, and {{print()}} should be mirrored
to the DataStream API. 
> The semantics of the calls are different. We need to be able to sample parts of a stream,
e.g. by supplying a time period in the arguments to the methods. Users should use the {{JobClient}}
to retrieve the results.
> {code:java}
> StreamExecutionEnvironment env = StramEnvironment.getStreamExecutionEnvironment();
> DataStream<DataType> streamData = env.addSource(..).map(..);
> JobClient jobClient = env.executeWithControl();
> Iterable<DataType> sampled = jobClient.sampleStream(streamData, Time.seconds(5));
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message