flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6928) Kafka sink: default topic should not need to exist
Date Thu, 15 Jun 2017 08:38:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050157#comment-16050157
] 

Tzu-Li (Gordon) Tai commented on FLINK-6928:
--------------------------------------------

I agree that the {{defaultTopic}} configuration is a bit "off" w.r.t. dynamic topics in the
{{KeyedSerializationSchema}}.

Taking a step back, it might make sense to:

1. Make the support for routing to multiple dynamic topics more prominent in the first-level
API of the producer.

2. Make setting the default an explicit option. e.g.
{code}
FlinkKafkaProducer prod = new FlinkKafkaProducer(serSchema, props, partitioner)
prod.setDefaultTopic("") // optional setting. If set, we fetch metadata for it in open.
{code}

What do you think?

> Kafka sink: default topic should not need to exist
> --------------------------------------------------
>
>                 Key: FLINK-6928
>                 URL: https://issues.apache.org/jira/browse/FLINK-6928
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector
>    Affects Versions: 1.3.0, 1.2.1
>            Reporter: Erik van Oosten
>
> When using a Kafka sink, the defaultTopic needs to exist even when it is never used.
It would be nice if fetching partition information for the default topic would be delayed
until the moment a topic is actually used.
> Cause: {{FlinkKafkaProducerBase.open}} fetches partition information for the default
topic.
> In addition, it would be nice if we could signal that the defaultTopic is not needed
by passing {{null}}. Currently, a value for the defaultTopic is required.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message