kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Craig Ching <craigch...@gmail.com>
Subject High CPU Usage on
Date Thu, 16 Jul 2015 13:41:50 GMT
Hi all,

We're experiencing high cpu usage on some of our development machines when
kafka should be idle.  These are single node kafka instances used only for
development, so far we haven't seen any problems on our production system.

It doesn't seem to happen until we deploy a storm topology and the more
topologies we deploy, the worse it gets.  And the higher the partitions for
the topics, the worse it gets.

For instance, on my system if I change the number of partitions from 3 to 1
for ~10 topics, I experience a decrease in CPU usage from ~89% to ~50%

I'll admit, I'm still getting up to speed on kafka, so maybe having 3
partitions on our development machines is crazy, but, still, even 1
partition causes this excessive CPU.

I have profiled using jvisualvm and the hotspot seems to be
RequestChannel.receiveRequest().  I've tinkered with the code a bit, but
haven't been able to affect any change.

Does anyone have any experience with such a scenario?  Is there anything I
can provide that would help troubleshoot this?

I appreciate any help!


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message