kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ewen Cheslack-Postava <e...@confluent.io>
Subject Re: latency performance test
Date Thu, 16 Jul 2015 04:21:21 GMT
The tests are meant to evaluate different things and the way they send
messages is the source of the difference.

EndToEndLatency works with a single message at a time. It produces the
message then waits for the consumer to receive it. This approach guarantees
there is no delay due to queuing. The goal with this test is to evaluate
the *minimum* latency.

ProducerPerformance focuses on achieving maximum throughput. This means it
will enqueue lots of records so it will always have more data to send (and
can use batching to increase the throughput). Unlike EndToEndLatency, this
means records may just sit in a queue on the producer for awhile because
the maximum number of in flight requests has been reached and it needs to
wait for responses for those requests. Since EndToEndLatency only ever has
one record outstanding, it will never encounter this case.

Batching itself doesn't increase the latency because it only occurs when
the producer is either a) already unable to send messages anyway or b)
linger.ms is greater than 0, but the tests use the default setting that
doesn't linger at all.

In your example for ProducerPerformance, you have 100 byte records and will
buffer up to 64MB. Given the batch size of 8K and default producer settings
of 5 in flight requests, you can roughly think of one round trip time
handling 5 * 8K = 40K bytes of data. If your roundtrip is 1ms, then if your
buffer is full at 64MB it will take you 64 MB / (40 KB/ms) = 1638ms = 1.6s.
That means that the record that was added at the end of the buffer had to
just sit in the buffer for 1.6s before it was sent off to the broker. And
if your buffer is consistently full (which it should be for
ProducerPerformance since it's sending as fast as it can), that means
*every* record waits that long.

Of course, these numbers are estimates, depend on my having used 1ms, but
hopefully should make it clear why you can see relatively large latencies.


On Wed, Jul 15, 2015 at 1:38 AM, Yuheng Du <yuheng.du.hust@gmail.com> wrote:

> Hi,
> I have run the end to end latency test and the producerPerformance test on
> my kafka cluster according to
> https://gist.github.com/jkreps/c7ddb4041ef62a900e6c
> In end to end latency test, the latency was around 2ms. In
> producerperformance test, if use batch size 8196 to send 50,000,000
> records:
> >bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance
> speedx1 50000000 100 -1 acks=1 bootstrap.servers=
> buffer.memory=67108864 batch.size=8196
> The results show that max latency is 3617ms, avg latency 626.7ms. I wanna
> know why the latency in producerperformance test is significantly larger
> than end to end test? Is it because of batching? Are the definitons of
> these two latencies different? I looked at the source code and I believe
> the latency is measure for the producer.send() function to complete. So
> does this latency includes transmission delay, transferring delay, and what
> other components?
> Thanks.
> best,
> Yuheng


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message