kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Koshy <jjkosh...@gmail.com>
Subject Re: Kafka Producer load distribution
Date Thu, 06 Mar 2014 01:04:23 GMT
> I am assuming this will reduce the inter-machine message transfer and will
> improve the performance.
> 
> My questions are :
> 
> 1) Does it really help in improving performance, when message is produced
> and consumed from same machine in a distributed environment.

I doubt that it helps a whole lot - especially if you let the producer
batch messages in a single request (default).

> 2) I read that producer can fetch metadata from broker about all
> leader-partition mapping for a topic. It will help to pick the leader
> present in the same machine as producer. How a producer can fetch this
> metadata ? Could not find any implementation.

You can use the SyncProducer or SimpleConsumer class - which provide a
send(<request>) API that can accept a topic metadata request and
returns a topic metadata response.

-- 
Joel

Mime
View raw message