flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "bupt_ljy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-12668) Introduce fromParallelElements for generating DataStreamSource
Date Wed, 29 May 2019 07:29:00 GMT
bupt_ljy created FLINK-12668:

             Summary: Introduce fromParallelElements for generating DataStreamSource
                 Key: FLINK-12668
                 URL: https://issues.apache.org/jira/browse/FLINK-12668
             Project: Flink
          Issue Type: Improvement
    Affects Versions: 1.8.0
            Reporter: bupt_ljy
            Assignee: bupt_ljy
             Fix For: 1.9.0

We've already have fromElements function in StreamExecutionEnvironment to generate a non-parallel
DataStreamSource. We should introduce a similar fromParallelElements function because:

1. The current implementations of ParallelSourceFunction are mostly bound to external resources
like kafka source. And we need a more lightweight parallel source function that can be easily
created. The SplittableIterator is too heavy by the way.
2. It's very useful if we want to verify or test something in a parallel processing environment.

This message was sent by Atlassian JIRA

View raw message