kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Weide Zhang <weo...@gmail.com>
Subject kafka message expiry question
Date Thu, 20 Nov 2014 07:26:42 GMT

as far as i understand, log retention time in kafka will delete message
that's older than the retention time. i'm wonder what it implies for
consumer since i'm using simple consumer to manage offsets in predefined
consumer group.

say i have a list of messages for a partition of topic:

1,2,3,4,5 are the message (offsets) associated with the partition in
current time.

if message 1,2,3 expired earlier and only 4,5 are left, does that mean
consumer can only consume 4,5 and need a way to detect 1,2,3 has expired
and make sure it never reads before the earliest offset for the partition ?

if all 1,2,3,4,5 are expired, it seems the offset will become 0. i assume
in this case consumer need to reset its consumed offsets to 0 for the
consumer group.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message