kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: kafka.SocketServerStats write rate metrics seem wrong
Date Wed, 06 Aug 2014 16:03:26 GMT
TotalBytesWritten is probably for sending the responses. You probably
should look at TotalBytesRead for incoming data rate.

Thanks,

Jun


On Tue, Aug 5, 2014 at 12:23 PM, Neil Harkins <nharkins@gmail.com> wrote:

> Hi.
> I'm seeing some odd numbers from kafka.SocketServerStats.
>
> Ideally, I'd like to have stats broken down per-topic, e.g.
> what's our most written/read topics? For write rates, I've got
> a separate process iterating topics every minute, doing this:
> (head_offset_now - head_offset_last) / (time_now - time_last)
> and inserting that into graphite (was previously inserting the
> head_offset, but graphite's derivative() on 2000+ topics
> was a bit much).
>
> a graphite sumSeries() across them shows 20~25 MB/s
> on a single-instance kafka 0.7.2, which sounds correct,
> but when I compare that to the kafka.SocketServerStats
> we're collecting in graphite, derivative(TotalBytesWritten)
> shows 3.0GB/s, and BytesWrittenPerSecond shows
> 200KB/s, neither of which jive with my offset rate.
>
> Is anyone aware of any weirdness with those stats?
>
> thanks in advance for any insight,
> -neil
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message