kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Palino <tpal...@gmail.com>
Subject Re: Last offset in all partitions
Date Wed, 06 Jul 2016 15:36:14 GMT
We do this through our monitoring agents by pulling it as a metric from the
LogEndOffset beans. By putting it into our metrics system we get a mapping
of timestamp to offset for every partition with (currently) 60 second
granularity. Useful for offset resets and other tasks.

-Todd

On Wednesday, July 6, 2016, Kristoffer Sjögren <stoffe@gmail.com> wrote:

> Hi
>
> Is there a way to get the last offset written by all partitions of a
> topic programmatically using the 0.10.0.0 API?
>
> At the moment I use KafkaConsumer.seekToEnd as seen in this gist[1]
> but maybe there is a better, more efficient, way to do it?
>
> Cheers,
> -Kristoffer
>
> [1] https://gist.github.com/krisskross/a49e462bedb89505e372672cd81129ab
>


-- 
*Todd Palino*
Staff Site Reliability Engineer
Data Infrastructure Streaming



linkedin.com/in/toddpalino

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message