kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neil Harkins <nhark...@gmail.com>
Subject kafka.SocketServerStats write rate metrics seem wrong
Date Tue, 05 Aug 2014 19:23:26 GMT
I'm seeing some odd numbers from kafka.SocketServerStats.

Ideally, I'd like to have stats broken down per-topic, e.g.
what's our most written/read topics? For write rates, I've got
a separate process iterating topics every minute, doing this:
(head_offset_now - head_offset_last) / (time_now - time_last)
and inserting that into graphite (was previously inserting the
head_offset, but graphite's derivative() on 2000+ topics
was a bit much).

a graphite sumSeries() across them shows 20~25 MB/s
on a single-instance kafka 0.7.2, which sounds correct,
but when I compare that to the kafka.SocketServerStats
we're collecting in graphite, derivative(TotalBytesWritten)
shows 3.0GB/s, and BytesWrittenPerSecond shows
200KB/s, neither of which jive with my offset rate.

Is anyone aware of any weirdness with those stats?

thanks in advance for any insight,

View raw message