kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Gustafson <ja...@confluent.io>
Subject Re: Checkpointing with custom metadata
Date Tue, 04 Aug 2015 17:11:00 GMT
I couldn't find a jira for this, so I added KAFKA-2403.

-Jason

On Tue, Aug 4, 2015 at 9:36 AM, Jay Kreps <jay@confluent.io> wrote:

> Hey James,
>
> You are right the intended use of that was to have a way to capture some
> very small metadata about your state at the time of offset commit in an
> atomic way.
>
> That field isn't exposed but we do need to add it to the new consumer api
> (I think just no one has done it yet.
>
> -Jay
>
> On Mon, Aug 3, 2015 at 1:52 PM, James Cheng <jcheng@tivo.com> wrote:
>
> > According to
> >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommitRequest
> ,
> > we can store custom metadata with our checkpoints. It looks like the high
> > level consumer does not support committing offsets with metadata, and
> that
> > in order to checkpoint with custom metadata, we have to issue the
> > OffsetCommitRequest ourselves. Is that correct?
> >
> > Thanks,
> > -James
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message