kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Posta <christian.po...@gmail.com>
Subject Re: Producer offset commit API
Date Mon, 16 May 2016 17:09:52 GMT
If you're using KafkaConnect, it does it for you!

basically you set the sourceRecord's "sourcePartition" and "sourceOffset"
fields (
https://github.com/christian-posta/kafka/blob/8db55618d5d5d5de97feab2bf8da4dc45387a76a/connect/api/src/main/java/org/apache/kafka/connect/source/SourceRecord.java#L51-51)
and Kafka will handle storing the offsets (default in Kafka i believe)
https://github.com/christian-posta/kafka/blob/1587aeed9a551c7c755666040fe26d066dba6857/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java#L198-198

If you need to read the offsets, just use the offsetStorageReader (
https://github.com/christian-posta/kafka/blob/417e283d643d8865aa3e79dffa373c8cc853d78f/connect/api/src/main/java/org/apache/kafka/connect/source/SourceTaskContext.java#L31-31)
which is automatically available to a source task.

On Mon, May 16, 2016 at 9:25 AM, Kanagha <er.kanagha@gmail.com> wrote:

> Hi,
>
> I am trying to find out the API for committing producer offset for Kafka
>
> I found this example:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka
>
> Which would work for committing consumer offsets. Is there a separate API
> for committing producer offset using Kafka connect?
>
> Thanks
> Kanagha
>
>
> --
> Kanagha
>



-- 
*Christian Posta*
twitter: @christianposta
http://www.christianposta.com/blog
http://fabric8.io

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message