kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: [kafka-dev] performance of async producer stops increase when broker is more than 2
Date Tue, 27 Mar 2012 19:25:36 GMT
Hi,

How many producers are you using and on how many machines? Also what is the
network speed (1gbit?) Likely you are maxing out either the producer
machine(s) or the network itself.

Also, quick question, how did you come across this address? This address
has been dead for a while now, and the new mailing list is on Apache so I
just want to make sure we don't have a dangling pointer somewhere. Would be
good to move the discussion to the Apache list so others can find it.

-Jay

On Tue, Mar 27, 2012 at 2:10 AM, mmLiu <diveintotomorrow@gmail.com> wrote:

> I tested the performance of producer with ./bin/kafka-producer-perf-test.sh
>
> when 1 broker:
> 20.3164 M/s,213033.3823#/s (async)
>
> when 2 brokers:
> 37.4754 M/s,392958.1892#/s (async)
>
> when 3 brokers(perf stops increase)
> 36.6925 M/s, 384748.5668#/s (async)
>
> when 4 brokers(even decrease):
> 33.2244 M/s, 348383.5006 #/s (async)
>
>
> Any idea why?
>
> FYI:the performance of sync producer increases(almost) linearly as number
> of brokers increases.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message