flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (Jira)" <j...@apache.org>
Subject [jira] [Created] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups
Date Sun, 19 Jan 2020 22:45:00 GMT
Stephan Ewen created FLINK-15670:
------------------------------------

             Summary: Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and
Flink's KeyGroups
                 Key: FLINK-15670
                 URL: https://issues.apache.org/jira/browse/FLINK-15670
             Project: Flink
          Issue Type: Improvement
          Components: API / DataStream, Connectors / Kafka
            Reporter: Stephan Ewen
             Fix For: 1.11.0


This Source/Sink pair would serve two purposes:

1. You can read topics that are already partitioned by key and process them without partitioning
them again (avoid shuffles)

2. You can use this to shuffle through Kafka, thereby decomposing the job into smaller jobs
and independent pipelined regions that fail over independently.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message