kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sönke Liebau <soenke.lie...@opencore.com.INVALID>
Subject Re: multiple threads calling kafka producer
Date Mon, 20 Nov 2017 09:04:43 GMT
Hi Ranjit,

there shouldn't be an issue with having multiple producers, but in general
the preferred approach is to share one producer across multiple threads
(see producer javadoc [1]) as this allows for more efficient pooling of
records and less network overhead.

Kind regards,
Sönke


[1]:
https://kafka.apache.org/10/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html


On Fri, Nov 17, 2017 at 7:23 PM, Ranjit Kumar <ranjitkum@gmail.com> wrote:

> Hi Manna,
>
> If needed i will close, it's sample one i mentioned, I am thinking of 2
> options like
> 1) keep pool of threads which will receive the data and send continuously
> using producer
> 2) when ever sending create a thread ,send data and close the thread (may
> be one point of time 300 threads also will create)
>
>
> Regards,
> Ranjit
>
> On Fri, Nov 17, 2017 at 11:47 PM, M. Manna <manmedia@gmail.com> wrote:
>
> > Are you planning to close these at any point? I'm not seeung any close()
> > calls.
> >
> > On 17 Nov 2017 6:09 pm, "Ranjit Kumar" <ranjitkum@gmail.com> wrote:
> >
> > Hi All,
> >
> > I am writing java program with thread pool may be  100 threads  and all
> > threads will have kafka producers
> > (org.apache.kafka.clients.producer.ProducerRecord) for the same topic,
> > will
> > their be any problem ?
> >
> >        thread-1
> >         ProducerRecord<String, String> data = new ProducerRecord<>(topic,
> > msg); // same topic
> >         stringProducer.send(data);
> >
> >        thread-2
> >         ProducerRecord<String, String> data = new ProducerRecord<>(topic,
> > msg); // same topic
> >         stringProducer.send(data);
> >
> >        thread-3
> >         ProducerRecord<String, String> data = new ProducerRecord<>(topic,
> > msg); // same topic
> >         stringProducer.send(data);
> >
> > Regards,
> > Ranjit
> >
>



-- 
Sönke Liebau
Partner
Tel. +49 179 7940878
OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message