kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abel ." <a...@amtech.mx>
Subject Re: Better strategy for sending a message to multiple topics
Date Mon, 21 Dec 2015 04:26:33 GMT

Jens, I would do it that way but different queues contain different
contents. That's why I need more than a single queue to solve my problem.
However, I have a hierarchical structure of queues (like a tree or forest)
and when you push a message to a queue I need to make it available to all
the ancestors in the tree of topics. That's why I need to push to multiple
queues. It's like the most straightforward approach but I haven't found
anything better.

Dong, I think I'm using asynchronous producing but I'm not completely sure.
I'm using an instance of the KafkaProducer object and calling the send
method which returns a Future object and I'm not calling the get method of
that Future instance so I'm assuming that's asynchronous, isn't it?

About the ACK I'm not setting it so I suppose that I'm using the default
which seems to be 1. Am I right?

Thanks both for your answers.


On Sun, Dec 20, 2015 at 10:35 PM, Dong Lin <lindong28@gmail.com> wrote:

> Did you try asynchronous produce? That should provide better performance
> because asynchronous produce doesn't block waiting for response.
> BTW, what is your ack setting for producer?
> Dong
> On Fri, Dec 18, 2015 at 1:16 PM, Abel . <abel@amtech.mx> wrote:
> > Hi,
> >
> > I have this scenario where I need to send a message to multiple topics. I
> > create a single KafkaProducer, prepare the payload and then I call the
> send
> > method of the producer for each topic with the correspoding
> ProducerRecord
> > for the topic and the fixed message. However, I have noticed that this
> > procedure takes some time depending on the number of topics. For
> instance,
> > to send a message to 30 topics it takes more than 3s because each request
> > takes about 100ms to return from the send method. Is there a better way
> to
> > accomplish this same task? Any recommendation?
> >
> > Regards,
> >
> > Abel.
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message