kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xinyi Su <xiny...@gmail.com>
Subject Re: Kafka producer perf script throw java.io.IOException
Date Tue, 10 Feb 2015 06:57:32 GMT
Hi, Jun
I create a Jira issue to track it. Please have a look at it.

https://issues.apache.org/jira/browse/KAFKA-1939

Thanks.
Xinyi

On 10 February 2015 at 10:38, Jun Rao <jun@confluent.io> wrote:

> The new producer uses a different built-in metrics package. Currently, it
> only supports a jmx reporter for the metrics. So you will have to get the
> metrics from jmx.
>
> We can add the csv reporter in ProducerPerformance for the new producer by
> using the new metrics api. Could you file a jira for that?
>
> Thanks,
>
> Jun
>
> On Wed, Feb 4, 2015 at 11:48 PM, Xinyi Su <xinyisu@gmail.com> wrote:
>
> > Hi,
> >
> > You are right. The file is created by Kafka and Kafka producer csv
> reporter
> > append metrics continuously during perf test. It should not be created
> > continuously during test but create once only.
> >
> > Another thing is when I turn on "--new-producer" and "--sync" option,
> > nothing is appended into  ProducerRequestSize.csv. From source code,  I
> > have not got any NewShinyProducer record stats code. It may be the
> reason.
> >
> > Thanks.
> > Xinyi
> >
> > On 5 February 2015 at 15:26, Jaikiran Pai <jai.forums2013@gmail.com>
> > wrote:
> >
> > > >> java.io.IOException: Unable to create /tmp/PerfTopic22_1/
> > > ProducerRequestSize.csv
> > >
> > > It looks like a file with that exact same name already exists which is
> > > causing that file creation request to fail. This indicates that
> probably
> > > the metric name (ProducerRequestSize) from which the file is created,
> is
> > > duplicate for whatever reason.
> > >
> > > -Jaikiran
> > >
> > >
> > > On Thursday 05 February 2015 12:49 PM, Xinyi Su wrote:
> > >
> > >> Hi,
> > >>
> > >> I need to get more metrics from csv reporter.  If turn off
> csv-reporter,
> > >> few output is shown.
> > >>
> > >> Thanks.
> > >> Xinyi
> > >>
> > >> On 5 February 2015 at 13:09, tao xiao <xiaotao183@gmail.com> wrote:
> > >>
> > >>  Hi,
> > >>>
> > >>> In order to get it work you can turn off csv-reporter.
> > >>>
> > >>> On Thu, Feb 5, 2015 at 1:06 PM, Xinyi Su <xinyisu@gmail.com>
wrote:
> > >>>
> > >>>  Hi,
> > >>>>
> > >>>> Today I updated Kafka cluster from 0.8.2-beta to 0.8.2.0 and run
> kafka
> > >>>> producer performance test.
> > >>>>
> > >>>> The test cannot continue because of some exceptions thrown which
> does
> > >>>> not
> > >>>> occur at 0.8.2-beta. My perf library is kafka-perf_2.9.2-0.8.0.jar
> > which
> > >>>>
> > >>> is
> > >>>
> > >>>> the latest version on maven repository.
> > >>>>
> > >>>> -bash-4.1$ bin/kafka-producer-perf-test.sh   --broker-list <broker
> > >>>> list>
> > >>>> --topics PerfTopic22 --sync --initial-message-id 1 --messages 200000
> > >>>> --csv-reporter-enabled --metrics-dir /tmp/PerfTopic22_1
> > >>>> --message-send-gap-ms 20 --request-num-acks -1 --batch-size 1
> > >>>>
> > >>>> java.io.IOException: Unable to create
> > >>>> /tmp/PerfTopic22_1/ProducerRequestSize.csv
> > >>>> at
> > >>>>
> > >>>>
> > >>>>  com.yammer.metrics.reporting.CsvReporter.createStreamForMetric(
> > >>> CsvReporter.java:141)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>>  com.yammer.metrics.reporting.CsvReporter.getPrintStream(
> > >>> CsvReporter.java:257)
> > >>>
> > >>>> at
> > >>>>
> > >>>
> > com.yammer.metrics.reporting.CsvReporter.access$000(CsvReporter.java:22)
> > >>>
> > >>>> at
> > >>>>
> > >>>>  com.yammer.metrics.reporting.CsvReporter$1.getStream(
> > >>> CsvReporter.java:156)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>>  com.yammer.metrics.reporting.CsvReporter.processHistogram(
> > >>> CsvReporter.java:194)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>>  com.yammer.metrics.reporting.CsvReporter.processHistogram(
> > >>> CsvReporter.java:22)
> > >>>
> > >>>> at com.yammer.metrics.core.Histogram.processWith(Histogram.java:231)
> > >>>> at
> com.yammer.metrics.reporting.CsvReporter.run(CsvReporter.java:163)
> > >>>> at
> > >>>>
> > >>>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > >>>
> > >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> > >>>> at
> > >>>>
> > >>>>
> > >>>>  java.util.concurrent.ScheduledThreadPoolExecutor$
> > >>> ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>>  java.util.concurrent.ScheduledThreadPoolExecutor$
> > >>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>>  java.util.concurrent.ThreadPoolExecutor.runWorker(
> > >>> ThreadPoolExecutor.java:1145)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>>  java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > >>> ThreadPoolExecutor.java:615)
> > >>>
> > >>>> at java.lang.Thread.run(Thread.java:745)
> > >>>>
> > >>>>
> > >>>
> > >>> --
> > >>> Regards,
> > >>> Tao
> > >>>
> > >>>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message