kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zoran <zoran.ljubi...@bulbtech.com>
Subject Re: Setting topic's offset from the shell
Date Mon, 05 Mar 2018 07:40:04 GMT
The procedure you have suggested is good for replaying everything from 
the very beginning, but I would like to replay messages from an 
arbitrary offset.

On the backend I have a ClickHouse table that listens Kafka topic with 
its group_id.

In case of problems between ClickHouse table and Kafka, I would like to 
replay messages that are missing from the ClickHouse table.

As I can't do anything on the ClickHouse side (there are just specified 
topic and group parameters), I need a mechanism to set the offset from 
the outside.

In order to minimize complexity of the operation, I would like to set 
the offset from Kafka shell scripts if possible.

On 02/28/2018 10:29 AM, UMESH CHAUDHARY wrote:
> You might want to set group.id config in kafka-console-consumer (or in any
> other consumer) to the value which you haven't used before. This will
> replay all available messages in the topic from start if you use
> --from-beginning in console consumer.
> On Wed, 28 Feb 2018 at 14:19 Zoran <zoran.ljubisic@bulbtech.com> wrote:
>> Hi,
>> If I have a topic that has been fully read by consumers, how to set the
>> offset from the shell to some previous value in order to reread again
>> several messages?
>> Regards.

View raw message