storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Yost <hokiege...@gmail.com>
Subject Virtually all time spent in lmax.disruptor.ProcessingSequenceBuffer with fieldsGrouping
Date Thu, 10 Dec 2015 17:07:26 GMT
Hi Everyone,

I am attempting to get fieldsGrouping to scale with no success.  When I use
localOrShuffleGrouping, I get throughput 10x faster than fieldsGrouping.
In addition, with fieldsGrouping, the topology eventually slows down
dramatically and tuples start to fail. Finally, when I use fieldsGrouping,
~99% of the time is spent in lmax.disruptor.ProcessingSequenceBuffer for
the Bolt that has proven to be the bottleneck, whereas with
localOrShuffleGrouping, 55-60%

I have tried changing the number of netty server threads, netty client
threads, the send and receive buffer sizes, and nothing is working.

Anyone have any thoughts on this?

Thanks

--John

Mime
View raw message