flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3398) Flink Kafka consumer should support auto-commit opt-outs
Date Fri, 08 Apr 2016 19:11:25 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15232739#comment-15232739

ASF GitHub Bot commented on FLINK-3398:

Github user shikhar commented on the pull request:

    Another thing is that presence of offsets in ZK (0.8) / Kafka (0.9) also affects what
happens when a job is starting without a checkpoint / savepoint. If they are present, the
reset mode won't be respected, and there is potential for dropping messages (e.g. if the recovery
strategy in the absence of a valid checkpoint or savepoint is to process from the beginning
of available data with offset reset mode 'earliest').

> Flink Kafka consumer should support auto-commit opt-outs
> --------------------------------------------------------
>                 Key: FLINK-3398
>                 URL: https://issues.apache.org/jira/browse/FLINK-3398
>             Project: Flink
>          Issue Type: Bug
>            Reporter: Shikhar Bhushan
> Currently the Kafka source will commit consumer offsets to Zookeeper, either upon a checkpoint
if checkpointing is enabled, otherwise periodically based on {{auto.commit.interval.ms}}
> It should be possible to opt-out of committing consumer offsets to Zookeeper. Kafka has
this config as {{auto.commit.enable}} (0.8) and {{enable.auto.commit}} (0.9).

This message was sent by Atlassian JIRA

View raw message