kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gwen Shapira <g...@confluent.io>
Subject Re: Checkpointing with custom metadata
Date Mon, 03 Aug 2015 22:17:44 GMT
You are correct. You can see that ZookeeperConsumerConnector is hardcoded
with null metadata.
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala#L310

More interesting, it looks like the Metadata is not exposed in the new
KafkaConsumer either.

Mind sharing what did you plan to use it for? this will help us figure out
how to expose it :)

Gwen


On Mon, Aug 3, 2015 at 1:52 PM, James Cheng <jcheng@tivo.com> wrote:

> According to
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommitRequest,
> we can store custom metadata with our checkpoints. It looks like the high
> level consumer does not support committing offsets with metadata, and that
> in order to checkpoint with custom metadata, we have to issue the
> OffsetCommitRequest ourselves. Is that correct?
>
> Thanks,
> -James
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message