beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Groh (JIRA)" <>
Subject [jira] [Assigned] (BEAM-2449) Support ValueProviders in KafkaIO
Date Wed, 14 Jun 2017 16:49:00 GMT


Thomas Groh reassigned BEAM-2449:

       Assignee:     (was: Thomas Groh)
    Component/s:     (was: runner-dataflow)

> Support ValueProviders in KafkaIO
> ---------------------------------
>                 Key: BEAM-2449
>                 URL:
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-extensions
>    Affects Versions: 2.0.0
>            Reporter: peay
>            Priority: Minor
> KafkaIO takes parameters at construction time:
> - When partitions are provided, KafkaIO actually does nothing particular at construction
> - When only a topic is provided, I believe KafkaIO does fetch the list of partitions
at construction time.
> Using ValueProvider for the brokers would be a good improvement. In particular, when
partitions are provided, a RuntimeProvider could be passed in.
> Being able to always use a RuntimeProvider would be even better, such as when using {{}}:
this would considerably improve the workflow of using Dataflow templates with KafkaIO, which
currently requires to hardcode brokers into the template.

This message was sent by Atlassian JIRA

View raw message