kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ewen Cheslack-Postava <e...@confluent.io>
Subject Re: Poor performance running performance test
Date Wed, 28 Jan 2015 08:25:13 GMT
That error indicates the broker closed the connection for some reason. Any
useful logs from the broker? It looks like you're using ELB, which could
also be the culprit. A connection timeout seems doubtful, but ELB can also
close connections for other reasons, like failed health checks.

-Ewen

On Tue, Jan 27, 2015 at 11:21 PM, Dillian Murphey <crackshotmule@gmail.com>
wrote:

> I was running the performance command from a virtual box server, so that
> seems like it was part of the problem.  I'm getting better results running
> this on a server on aws, but that's kind of expected.  Can you look at
> these results, and comment on the occasional warning I see?  I appreciate
> it!
>
> 1220375 records sent, 243928.6 records/sec (23.26 MB/sec), 2111.5 ms avg
> latency, 4435.0 max latency.
> 1195090 records sent, 239018.0 records/sec (22.79 MB/sec), 2203.1 ms avg
> latency, 4595.0 max latency.
> 1257165 records sent, 251433.0 records/sec (23.98 MB/sec), 2172.6 ms avg
> latency, 4525.0 max latency.
> 1230981 records sent, 246196.2 records/sec (23.48 MB/sec), 2173.5 ms avg
> latency, 4465.0 max latency.
> [2015-01-28 07:19:07,274] WARN Error in I/O with
> <myawsloadbalancer>(org.apache.kafka.common.network.Selector)
> java.io.EOFException
> at
>
> org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62)
> at org.apache.kafka.common.network.Selector.poll(Selector.java:248)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
> at java.lang.Thread.run(Thread.java:745)
> 1090689 records sent, 218137.8 records/sec (20.80 MB/sec), 2413.6 ms avg
> latency, 4829.0 max latency.
>
> On Tue, Jan 27, 2015 at 7:37 PM, Ewen Cheslack-Postava <ewen@confluent.io>
> wrote:
>
> > Where are you running ProducerPerformance in relation to ZK and the Kafka
> > brokers? You should definitely see much higher performance than this.
> >
> > A couple of other things I can think of that might be going wrong: Are
> all
> > your VMs in the same AZ? Are you storing Kafka data in EBS or local
> > ephemeral storage? If EBS, have you provisioned enough IOPS.
> >
> >
> > On Tue, Jan 27, 2015 at 4:29 PM, Dillian Murphey <
> crackshotmule@gmail.com>
> > wrote:
> >
> > > I'm a new user/admin to kafka. I'm running a 3 node ZK and a 6 brokers
> on
> > > aws.
> > >
> > > The performance I'm seeing is shockingly bad. I need some advice!
> > >
> > > bin/kafka-run-class.sh
> org.apache.kafka.clients.tools.ProducerPerformance
> > > test2 50000000 100 -1 acks=1 bootstrap.servers=5<myloadbalancer>:9092
> > > buffer.memory=67108864 batch.size=8196
> > >
> > >
> > >
> > >
> > > 6097 records sent, 13198.3 records/sec (1.26 MB/sec), 2098.0 ms avg
> > > latency, 4306.0 max latency.
> > > 71695 records sent, 14339.0 records/sec (1.37 MB/sec), 6658.1 ms avg
> > > latency, 9053.0 max latency.
> > > 65195 records sent, 13028.6 records/sec (1.24 MB/sec), 11504.0 ms avg
> > > latency, 13809.0 max latency.
> > > 71955 records sent, 14391.0 records/sec (1.37 MB/sec), 16137.4 ms avg
> > > latency, 18541.0 max latency.
> > >
> > > Thanks for any help!
> > >
> >
> >
> >
> > --
> > Thanks,
> > Ewen
> >
>



-- 
Thanks,
Ewen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message