kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: Producer thread-safety and performance
Date Wed, 04 Jan 2012 19:30:18 GMT
The producer is thread safe.

Whether or not you will see throughput advantages from having multiple
producer instances depends on your usage, you will have to try it out. For
larger messages or using the producer in async mode, one producer can
pretty easily saturate the network and adding more producers won't add much
in the way of throughput. For small messages sent in sync mode having more
than one may improve throughput.

-Jay

2012/1/4 Patricio Echag├╝e <patricioe@gmail.com>

> Hi all, I'm trying to get familiar with Kafka code base and scala. I
> haven't been to answer the question myself yet.
>
> I have a bunch of threads (managed by a Thread pool executor in Java) and
> would like to know/ask recommendation as far as thread-safety goes on the
> producer side.
>
> Should I have one producer object per thread ? or one for all ?
>
> I read in an old email that the producer has proper synchronization, but
> It's not clear if it follows the concurrent approach similar to
> java.concurrent package style or if it's simple java synchronization. My
> goal is to not slow down the message generation.
>
> Thanks in advance.
> Patricio
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message