flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leonid Ilyevsky (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-12548) FlinkKafkaConsumer issues configuring underlying KafkaConsumer
Date Fri, 17 May 2019 20:54:00 GMT
Leonid Ilyevsky created FLINK-12548:
---------------------------------------

             Summary: FlinkKafkaConsumer issues configuring underlying KafkaConsumer
                 Key: FLINK-12548
                 URL: https://issues.apache.org/jira/browse/FLINK-12548
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Kafka
    Affects Versions: 1.8.0
            Reporter: Leonid Ilyevsky
         Attachments: KafkaError.txt

I observe two issues that are possibly related, or at least belong to the same part of the
code.

I am running multiple instances of pipelines (by setting parallelism) under Flink cluster.

First problem I see is the "Error registering AppInfo mbean", see attached stack trace [^KafkaError.txt] .
From the trace I understand that it tries to use consumer group id as the bean name; obviously
this fails as I have multiple instances using the same group id under the same task manager.

Second problem is, the client id is not set at all (I do provide the "client.id" property
when invoking FlinkKafkaConsumer constructor). This creates problems tracking consumers on
Kafka side. On the other hand, I obviously do not want to set the same client id on all instances
- preferably Flink should add maybe some UUID suffix to it or something to make it unique.

The same unique name could be used for that mbean, this way solving the first problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message