flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6928) Kafka sink: default topic should not need to exist
Date Thu, 15 Jun 2017 08:38:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050157#comment-16050157

Tzu-Li (Gordon) Tai commented on FLINK-6928:

I agree that the {{defaultTopic}} configuration is a bit "off" w.r.t. dynamic topics in the

Taking a step back, it might make sense to:

1. Make the support for routing to multiple dynamic topics more prominent in the first-level
API of the producer.

2. Make setting the default an explicit option. e.g.
FlinkKafkaProducer prod = new FlinkKafkaProducer(serSchema, props, partitioner)
prod.setDefaultTopic("") // optional setting. If set, we fetch metadata for it in open.

What do you think?

> Kafka sink: default topic should not need to exist
> --------------------------------------------------
>                 Key: FLINK-6928
>                 URL: https://issues.apache.org/jira/browse/FLINK-6928
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector
>    Affects Versions: 1.3.0, 1.2.1
>            Reporter: Erik van Oosten
> When using a Kafka sink, the defaultTopic needs to exist even when it is never used.
It would be nice if fetching partition information for the default topic would be delayed
until the moment a topic is actually used.
> Cause: {{FlinkKafkaProducerBase.open}} fetches partition information for the default
> In addition, it would be nice if we could signal that the defaultTopic is not needed
by passing {{null}}. Currently, a value for the defaultTopic is required.

This message was sent by Atlassian JIRA

View raw message