flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Flink Kafka Unused Properties Log Warnings
Date Wed, 01 Jun 2016 18:12:05 GMT
Hi Kim,

I agree with you. We should definitively filter out custom Flink
properties. I've filed a JIRA for this:
https://issues.apache.org/jira/browse/FLINK-4004
Thanks for the report!

Regards,
Robert


On Wed, Jun 1, 2016 at 6:42 PM, David Kim <david.kim@braintreepayments.com>
wrote:

> Hello!
>
> Using Flink 1.0.3.
>
> This is cosmetic but will help clean up logging I think.
>
> The Apache *KafkaConsumer* logs a warning [1] for any unused properties.
> This is great in case the developer has a typo or should clean up any
> unused keys.
>
> Flink's Kafka consumer and producer have some custom properties and it
> passes the entire properties map into the Apache Kafka consumer and
> producer. Custom properties such as "flink.poll-timeout" make it through
> and it triggers a warning.
>
> For the sake of cleaning up logging and removing false positives, I think
> it might make sense for the FlinkKafkaConsumer09 [2] and
> FlinkKafkaProducerBase [3] to filter out custom flink properties.
>
> Let me know if this sounds reasonable!
>
> Thanks!
> David
>
> [1]
> https://github.com/apache/kafka/blob/0.9.0/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L639
> [2]
> https://github.com/apache/flink/blob/release-1.0.3-rc3/flink-streaming-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.java#L186
> [3]
> https://github.com/apache/flink/blob/release-1.0.3-rc3/flink-streaming-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducerBase.java#L153
>
>
>

Mime
View raw message