beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amit Sela (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-549) SparkRunner should support Beam's KafkaIO instead of providing it's own.
Date Fri, 12 Aug 2016 21:29:20 GMT
Amit Sela created BEAM-549:
------------------------------

             Summary: SparkRunner should support Beam's KafkaIO instead of providing it's
own.
                 Key: BEAM-549
                 URL: https://issues.apache.org/jira/browse/BEAM-549
             Project: Beam
          Issue Type: Bug
          Components: runner-spark
            Reporter: Amit Sela
            Assignee: Amit Sela


For portability, and in the spirit of Apache Beam, the SparkRunner should use the Beam implementation
of KafkaIO instead of it's own.

Having said that, it might choose to translate the KafkaIO as defined in the pipeline into
it's own internal implementation, but should still map the properties the user defined in
the pipeline in a way that the IO behaves the same - i.e., brokers, topic, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message