kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jorge Rodriguez <jo...@bloomreach.com>
Subject Spikes in kafka bytes out (while bytes in remain the same)
Date Mon, 11 Apr 2016 20:54:03 GMT
We are running a kafka cluster for our real-time pixel processing
pipeline.  The data is produced from our pixel servers into kafka, and then
consumed by a spark streaming application.  Based on this, I would expect
that the bytes in vs bytes out should be roughly equal, as each message
should be consumed once.

Under normal operations, the bytes out is a little less than 2X the bytes
in.  Does anyone know why this is?  We do use a replication factor of 2.

Occasionally, we get a spike in Bytes out.  But bytes in remain the same
(see image below).  This correlates with a significant delay in processing
time in the spark streaming side.

Below is a chart of kafka reported bytes out vs in.  The system level
network metrics show the same information (transferred bytes spike).

Could anyone provide some tips for debugging/getting to the bottom of this
issue?

Thanks,
Jorge

*Kafka reported Bytes in Per topic and for all topics vs Kafka bytes out:*

[image: Inline image 1]

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message