flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Kim <david....@braintreepayments.com>
Subject Flink Kafka Unused Properties Log Warnings
Date Wed, 01 Jun 2016 16:42:43 GMT
Hello!

Using Flink 1.0.3.

This is cosmetic but will help clean up logging I think.

The Apache *KafkaConsumer* logs a warning [1] for any unused properties.
This is great in case the developer has a typo or should clean up any
unused keys.

Flink's Kafka consumer and producer have some custom properties and it
passes the entire properties map into the Apache Kafka consumer and
producer. Custom properties such as "flink.poll-timeout" make it through
and it triggers a warning.

For the sake of cleaning up logging and removing false positives, I think
it might make sense for the FlinkKafkaConsumer09 [2] and
FlinkKafkaProducerBase [3] to filter out custom flink properties.

Let me know if this sounds reasonable!

Thanks!
David

[1]
https://github.com/apache/kafka/blob/0.9.0/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L639
[2]
https://github.com/apache/flink/blob/release-1.0.3-rc3/flink-streaming-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.java#L186
[3]
https://github.com/apache/flink/blob/release-1.0.3-rc3/flink-streaming-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducerBase.java#L153

Mime
View raw message