flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3398) Flink Kafka consumer should support auto-commit opt-outs
Date Mon, 14 Mar 2016 14:34:33 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193372#comment-15193372
] 

ASF GitHub Bot commented on FLINK-3398:
---------------------------------------

Github user rmetzger commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1690#discussion_r56009318
  
    --- Diff: flink-streaming-connectors/flink-connector-kafka-0.8/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer08.java
---
    @@ -295,7 +296,7 @@ public void open(Configuration parameters) throws Exception {
     
     	@Override
     	public void run(SourceContext<T> sourceContext) throws Exception {
    -		if (fetcher != null) {
    +		if (fetcher != null && isAutoCommitEnabled()) {
    --- End diff --
    
    This is not correct!
    If the fetcher != null and auto commit disabled, `fetcher.run()` (line 316) is never called!
This means that the fetcher is doing anything at all!
    
    I'm not sure how the integration tests were passing with that change! (maybe there is
no test testing auto commit enable = false.)


> Flink Kafka consumer should support auto-commit opt-outs
> --------------------------------------------------------
>
>                 Key: FLINK-3398
>                 URL: https://issues.apache.org/jira/browse/FLINK-3398
>             Project: Flink
>          Issue Type: Bug
>            Reporter: Shikhar Bhushan
>
> Currently the Kafka source will commit consumer offsets to Zookeeper, either upon a checkpoint
if checkpointing is enabled, otherwise periodically based on {{auto.commit.interval.ms}}
> It should be possible to opt-out of committing consumer offsets to Zookeeper. Kafka has
this config as {{auto.commit.enable}} (0.8) and {{enable.auto.commit}} (0.9).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message