kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayuresh Gharat <gharatmayures...@gmail.com>
Subject Re: Kafka High Level Consumer Message Loss?
Date Mon, 13 Jul 2015 03:42:34 GMT
Can you confirm that you are not actually seeing the messages on the
lagging broker ?
Because if the Max Lag is 0 it should mean that consumer has read the
offsets till log end offset of the broker.

Thanks,

Mayuresh

On Fri, Jul 10, 2015 at 8:29 PM, Allen Wang <awang@netflix.com.invalid>
wrote:

> We have two applications that consume all messages from one Kafka cluster.
> We found that the MessagesPerSec metric started to diverge after some time.
> One of them matches the MessagesInPerSec metric from the Kafka broker,
> while the other is lower than the broker metric and appears to have some
> message loss.
>
> Both of them have the same OwnedPartitionsCount.
>
> Both of them have 0 MaxLag.
>
> How is that possible? Anything we should look at? Is the MaxLag metric not
> telling the truth?
>
> Thanks,
> Allen
>



-- 
-Regards,
Mayuresh R. Gharat
(862) 250-7125

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message