kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhavesh Mistry <mistry.p.bhav...@gmail.com>
Subject Re: Sending Same Message to Two Topics on Same Broker Cluster
Date Tue, 21 Oct 2014 16:14:44 GMT
Hi Neha,

All, I am saying is that if same byte[] or data  has to go to two topics
then, I have to call send twice and with same data has to transfer over the
wire twice (assuming the partition is on same broker for two topics, then
it not efficient.).  If Kafka Protocol allows to set multiple topics and
partitions for request then it would me great.
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-ProduceRequest
*ProducerRecord
<http://people.apache.org/~nehanarkhede/kafka-0.9-producer-javadoc/doc/org/apache/kafka/clients/producer/ProducerRecord.html#ProducerRecord(java.lang.String,
byte[], byte[])>*(java.lang.String *topic*, byte[] key, byte[] value)


Thanks,

Bhavesh

On Tue, Oct 21, 2014 at 8:26 AM, Neha Narkhede <neha.narkhede@gmail.com>
wrote:

> I'm not sure I understood your concern about invoking send() twice, once
> with each topic. Are you worried about the network overhead? Whether Kafka
> does this transparently or not, sending messages to different topics will
> carry some overhead. I think the design of the API is much more intuitive
> and cleaner if a message is sent to a topic partition.
>
> On Mon, Oct 20, 2014 at 9:17 PM, Bhavesh Mistry <
> mistry.p.bhavesh@gmail.com>
> wrote:
>
> > Hi Neha,
> >
> > Yes, I understand that but when transmitting single message (I can not
> set
> > List of all topics)  Only Single one.  So I will to add same message in
> > buffer with different topic. If Kakfa protocol, allows to add multiple
> > topic then message does not have to be re-transmited over the wire to add
> > to multiple topic.
> >
> > The Producer record only allow one topic.
> >
> >
> http://people.apache.org/~nehanarkhede/kafka-0.9-producer-javadoc/doc/org/apache/kafka/clients/producer/ProducerRecord.html
> >
> > Thanks for your quick response and I appreciate your help.
> >
> > Thanks,
> >
> > Bhavesh
> >
> >
> > On Mon, Oct 20, 2014 at 9:10 PM, Neha Narkhede <neha.narkhede@gmail.com>
> > wrote:
> >
> > > Not really. You need producers to send data to Kafka.
> > >
> > > On Mon, Oct 20, 2014 at 9:05 PM, Bhavesh Mistry <
> > > mistry.p.bhavesh@gmail.com>
> > > wrote:
> > >
> > > > Hi Kakfa Team,
> > > >
> > > >
> > > > I would like to send a single message to multiple topics (two for
> now)
> > > > without re-transmitting the message from producer to brokers.  Is
> this
> > > > possible?
> > > >
> > > > Both Producers Scala and Java does not allow this.   I do not have to
> > do
> > > > this all the time only based on application condition.
> > > >
> > > >
> > > > Thanks in advance of your help !!
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Bhavesh
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message