kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liquan Pei <liquan...@gmail.com>
Subject Re: Kafka Newbie question
Date Sat, 09 Apr 2016 08:12:06 GMT
Hi Pradeep,

Can you try to set enable.auto.commit = false if you want to read to the
earliest offset? According to the documentation, auto.offset.reset controls
what to do when there is no initial offset in Kafka or if the current
offset does not exist any more on the server (e.g. because that data has
been deleted). In case that auto commit is enabled, the committed offset is
available in some servers.


On Fri, Apr 8, 2016 at 10:44 PM, Pradeep Bhattiprolu <pbhattip9@gmail.com>

> Hi All
> I am a newbie to kafka. I am using the new Consumer API in a thread acting
> as a consumer for a topic in Kafka.
> For my testing and other purposes I have read the queue multiple times
> using console-consumer.sh script of kafka.
> To start reading the message from the beginning in my java code , I have
> set the value of the auto.offset.reset to "earliest".
> However that property does not guarantee that i begin reading the messages
> from start, it goes by the most recent smallest offset for the consumer
> group.
> Here is my question,
> Is there a assured way of starting to read the messages from beginning from
> Java based Kafka Consumer ?
> Once I reset one of my consumers to zero, do i have to do offset management
> myself for other consumer threads or does kafka automatically lower the
> offset to the first threads read offset ?
> Any information / material pointing to the solution are highly appreciated.
> Thanks
> Pradeep

Liquan Pei
Software Engineer, Confluent Inc

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message