kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrej Vladimirovich <udodizdu...@gmail.com>
Subject Re: kafka-producer-perf-test.sh - 0.8.2.1
Date Fri, 08 Jan 2016 21:24:58 GMT
Thanks Ewen. Do you know if kafka-run-class.sh
org.apache.kafka.clients.tools.ProducerPerformance
is single threaded? Or is there any way to specify number of threads?

On Fri, Jan 8, 2016 at 1:24 PM, Ewen Cheslack-Postava <ewen@confluent.io>
wrote:

> Ah, sorry, I missed the version number in your title. I think this tool saw
> some rearrangement in 0.9.0 and I was looking at the latest version.
> Unfortunately it doesn't look like the old kafka.tools.ProducerPerformance
> that is used in kafka-producer-perf-test.sh in 0.8.2.1 supports passing in
> additional properties.
>
> -Ewen
>
> On Fri, Jan 8, 2016 at 9:10 AM, Andrej Vladimirovich <
> udodizdupla@gmail.com>
> wrote:
>
> > Ewen,
> >
> > I tried that before like this:
> >
> > ./kafka-producer-perf-test.sh --broker-list test:9092 --topics test8-3
> > --messages 2000000 --new-producer --message-size 2000000
> > --show-detailed-stats max.request.size=10000000
> >
> > and it does not work. It comletly ignore this option.
> >
> > And --producer-props is not a valid option for
> kafka-producer-perf-test.sh.
> > Maybe it is not the right syntax? But I tried a lot of different ways and
> > have yet to find the right one.
> >
> > Thanks!
> >
> > Andrew
> >
> > On Fri, Jan 8, 2016 at 10:54 AM, Ewen Cheslack-Postava <
> ewen@confluent.io>
> > wrote:
> >
> > > Andrew,
> > >
> > > kafka-producer-perf-test.sh is just a wrapper around
> > > orga.apache.kafka.clients.tools.ProducerPerformance and all command
> line
> > > options should be forwarded. Can you just pass a --producer-props to
> set
> > > max.request.size to a larger value?
> > >
> > > -Ewen
> > >
> > > On Fri, Jan 8, 2016 at 7:51 AM, Andrej Vladimirovich <
> > > udodizdupla@gmail.com>
> > > wrote:
> > >
> > > > Hi!
> > > >
> > > > I am testing Kafka's performance with large messages and would like
> to
> > > > specify maximum request size when I run kafka-producer-perf-test.sh:
> > > >
> > > > ./kafka-producer-perf-test.sh --broker-list "test1:9092" --topics
> > test8-3
> > > > --messages 1000000 --new-producer --msage-size 1100000
> > > > --show-detailed-stats
> > > >
> > > > I always get this message if I specify somethine larger than 1MB:
> > > >
> > > > ERROR Error when sending message to topic test8-3 with key: 1 bytes,
> > > value:
> > > > 1100000 bytes with error: The message is 1100027 bytes when
> serialized
> > > > which is larger than the maximum request size you have configured
> with
> > > the
> > > > max.request.size configuration.
> > > > (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
> > > >
> > > > I know I can specify maximum request size with kafka-run-class.sh
> > > > org.apache.kafka.clients.tools.ProducerPerformance but I would like
> to
> > > use
> > > > kafka-producer-perf-test.sh if possible.
> > > >
> > > > Thanks!
> > > >
> > > > Andrew
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Ewen
> > >
> >
>
>
>
> --
> Thanks,
> Ewen
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message