kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Changing the Producer Config at runtime and TCP connections
Date Thu, 01 May 2014 04:54:21 GMT
Each producer has its own TCP connections. The socket related configs for
the producer are listed in
http://kafka.apache.org/documentation.html#producerconfigs

Currently, we don't support changing the configs in the producer
dynamically. You will have to create new producers to pick up config
changes.

Thanks,

Jun


On Wed, Apr 30, 2014 at 6:25 PM, Bhavesh Mistry
<mistry.p.bhavesh@gmail.com>wrote:

> Hi Kafka User and Dev team,
>
>
>
> I do not see any API to change the producer configuration dynamically at
> run-time after the producer object has been created.   We have situation
> where our pool is deployed in private cloud (open stack VMs), and we need
> to manage the lifecycle of the Producer respect to configuration change.
> Because one pool of machines ( eg search vs payment vs checkout) might have
> different configuration for same topic and we manage this using internal
> tool Cloud Configuration Management.  Basically, this software notifies us
> when there is configuration changed happen for the producer.   This is need
> in beginning when we are tuning new pool.  Basically we need to be able to
> restart producer without losing data (we use async with batch mode).
>
>
>
> Also, we have 4 producers connected to same kafka brokers within one JVM
> and have true round robin (write to each partition via event count % number
> of partition),  does Kafka Producer maintain TCP connections per Producer
> object or does Producer have global set of TCP connections shared among
> producers ?
>
>
>
> Majority of our Producers will be on VMs (open stack), is there any TCP
> tuning you guys recommend ?
>
>
>
> Please let us know answers to questions.
>
>
>
> We are still on Kafka 0.8.
>
>
>
> Thanks,
>
>
>
> Bhavesh
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message