kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: Sending Same Message to Two Topics on Same Broker Cluster
Date Tue, 21 Oct 2014 16:48:47 GMT
Hey Bhavesh,

This would only work if both topics happened to be on the same machine,
which generally they wouldn't.

-Jay

On Tue, Oct 21, 2014 at 9:14 AM, Bhavesh Mistry <mistry.p.bhavesh@gmail.com>
wrote:

> Hi Neha,
>
> All, I am saying is that if same byte[] or data  has to go to two topics
> then, I have to call send twice and with same data has to transfer over the
> wire twice (assuming the partition is on same broker for two topics, then
> it not efficient.).  If Kafka Protocol allows to set multiple topics and
> partitions for request then it would me great.
>
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-ProduceRequest
> *ProducerRecord
> <
> http://people.apache.org/~nehanarkhede/kafka-0.9-producer-javadoc/doc/org/apache/kafka/clients/producer/ProducerRecord.html#ProducerRecord(java.lang.String
> ,
> byte[], byte[])>*(java.lang.String *topic*, byte[] key, byte[] value)
>
>
> Thanks,
>
> Bhavesh
>
> On Tue, Oct 21, 2014 at 8:26 AM, Neha Narkhede <neha.narkhede@gmail.com>
> wrote:
>
> > I'm not sure I understood your concern about invoking send() twice, once
> > with each topic. Are you worried about the network overhead? Whether
> Kafka
> > does this transparently or not, sending messages to different topics will
> > carry some overhead. I think the design of the API is much more intuitive
> > and cleaner if a message is sent to a topic partition.
> >
> > On Mon, Oct 20, 2014 at 9:17 PM, Bhavesh Mistry <
> > mistry.p.bhavesh@gmail.com>
> > wrote:
> >
> > > Hi Neha,
> > >
> > > Yes, I understand that but when transmitting single message (I can not
> > set
> > > List of all topics)  Only Single one.  So I will to add same message in
> > > buffer with different topic. If Kakfa protocol, allows to add multiple
> > > topic then message does not have to be re-transmited over the wire to
> add
> > > to multiple topic.
> > >
> > > The Producer record only allow one topic.
> > >
> > >
> >
> http://people.apache.org/~nehanarkhede/kafka-0.9-producer-javadoc/doc/org/apache/kafka/clients/producer/ProducerRecord.html
> > >
> > > Thanks for your quick response and I appreciate your help.
> > >
> > > Thanks,
> > >
> > > Bhavesh
> > >
> > >
> > > On Mon, Oct 20, 2014 at 9:10 PM, Neha Narkhede <
> neha.narkhede@gmail.com>
> > > wrote:
> > >
> > > > Not really. You need producers to send data to Kafka.
> > > >
> > > > On Mon, Oct 20, 2014 at 9:05 PM, Bhavesh Mistry <
> > > > mistry.p.bhavesh@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Kakfa Team,
> > > > >
> > > > >
> > > > > I would like to send a single message to multiple topics (two for
> > now)
> > > > > without re-transmitting the message from producer to brokers.  Is
> > this
> > > > > possible?
> > > > >
> > > > > Both Producers Scala and Java does not allow this.   I do not have
> to
> > > do
> > > > > this all the time only based on application condition.
> > > > >
> > > > >
> > > > > Thanks in advance of your help !!
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > >
> > > > > Bhavesh
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message