kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Koshy <jjkosh...@gmail.com>
Subject Re: Kafka Producer in a high volume environment
Date Fri, 13 Apr 2012 22:50:38 GMT
> We have a java web application that receives approximately 1000 requests
> per second. All of the requests will send a message to Kafka.
> We were thinking of having a single instance of the Producer class in the
> application. Will that work?

In general, you would want to enable use the producer in async mode and
batch messages.

> The documentation says that "Internally, the producer maintains an elastic
> pool of connections to the brokers, one per broker".
> That means if we have one broker, we will only have one connection. How do
> we make sure that connections do not throttle the message flow? Do we have
> to create a pool of producers ourselves?

The underlying sync producer (which actually sends out the batches) has one
connection to the broker. You could of course create multiple producers but
you probably don't need that if you batch messages.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message