kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Loddengaard <a...@confluent.io>
Subject Re: Multiple producer threads in Kafka 0.9 MirrorMaker.
Date Wed, 03 Feb 2016 17:43:26 GMT
Hi Tao,

You're right -- thanks for correcting me. Only one producer is created, and
--num.streams controls the number of consumer streams. As such, to get more
producers you'd need to run multiple instances of MirrorMaker.

Thanks, Tao!

Alex

On Tue, Feb 2, 2016 at 7:20 PM, tao xiao <xiaotao183@gmail.com> wrote:

> Hi Alex,
>
> I doubt this is the case for producer. In 0.9 mirror maker there is only
> one producer created regardless of --num.streams. --num.streams only
> controls how many consumers are created.
>
>
> https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/MirrorMaker.scala#L201
>
>
> On Wed, 3 Feb 2016 at 09:53 Alex Loddengaard <alex@confluent.io> wrote:
>
> > Hi Tushar,
> >
> > In 0.9, MirrorMaker is now configured with a number of streams, not a
> > number of consumers and producers like it was in 0.8. Each stream has one
> > producer and one consumer. You can configure the number of streams
> > with --num.streams. So setting --num.streams 10 will create 10 producer
> > threads (and 10 consumers in the same threads).
> >
> > Alex
> >
> > On Tue, Feb 2, 2016 at 11:49 AM, Mhaskar, Tushar <
> > tmhaskar@paypal.com.invalid> wrote:
> >
> > > Hi All,
> > >
> > > I see that the —num.producers is removed from the Kafka 0.9
> MirrorMaker.
> > > Why is this?
> > > How can we create multiple producer threads for publishing in
> > MirrorMaker?
> > >
> > > Thanks,
> > > Tushar
> > >
> >
> >
> >
> > --
> > *Alex Loddengaard | **Solutions Architect | Confluent*
> > *Download Apache Kafka and Confluent Platform: www.confluent.io/download
> > <http://www.confluent.io/download>*
> >
>



-- 
*Alex Loddengaard | **Solutions Architect | Confluent*
*Download Apache Kafka and Confluent Platform: www.confluent.io/download
<http://www.confluent.io/download>*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message