kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: slow producing on ec2
Date Tue, 10 Jan 2012 22:52:36 GMT
Pierre-Yves,

I assume that this is a producer problem. If so, first, make sure flush
interval is not too small. Second, you can enable debug logging in
FileMessageSet to see log flush time. Then you can see if it's higher than
what you expect.

Thanks,

Jun


On Tue, Jan 10, 2012 at 2:19 PM, Pierre-Yves Ritschard <pyr@spootnik.org>wrote:

> Hi,
>
> One of my cluster runs on ec2 and I noticed that I have rather slow
> throughput, less than 500 per second. I isolated the trouble on the
> following setup
>
> 1 producer instance (m1.large)
> 5 consumer instances also running a broker (m2.xlarge)
> 1 zookeeper instance (m1.large)
>
> I am using a sync producer and have 60 partitions.
>
> On my local box, with everything running I would several thousand per
> second.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message