kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hans Jespersen <h...@confluent.io>
Subject Re: Producer Async Issue
Date Sat, 27 May 2017 22:22:00 GMT
So you just want to not block and just silently throw away the messages and lose them forever?
Kafka's persistence is all in the broker so there is no client side storing of data on disk.
The client will send in async mode until the client memory buffer is full. Only then does
it block, and this is by design because then it's up to your app to decide to either throw
the messages away, stop publishing, or store them somewhere outside of Kafka.

The easiest solution would be to run more than one broker so that they are fault tolerant
and will take over for any failed broker nodes.

-hans

> On May 27, 2017, at 12:40 PM, Abhimanyu Nagrath <abhimanyunagrath@gmail.com> wrote:
> 
> HI Hans,
> 
> What exactly I meant by asynchronous is that when my Kafka broker is down
> and I am trying to produce the message . It is getting stuck till the
> configured max.block.ms and after that further code is executed. What I am
> looking for is that whether the broker is down or not it should not get
> stuck.
> 
> 
> 
> Regards,
> Abhimanyu
> 
>> On Sat, May 27, 2017 at 10:30 PM, Hans Jespersen <hans@confluent.io> wrote:
>> 
>> The producer is asynchronous (assuming you mean the Java Producer)
>> 
>> https://kafka.apache.org/0102/javadoc/index.html?org/apache/
>> kafka/clients/producer/KafkaProducer.html
>> 
>> -hans
>> 
>>> On May 27, 2017, at 5:15 AM, Abhimanyu Nagrath <
>> abhimanyunagrath@gmail.com> wrote:
>>> 
>>> Hi,
>>> I am using Kafka 0.10.2 single node cluster and I want to know
>>> Kafka producer completely asynchronous. So what configuration I need to
>>> change in order to make producer completely asynchronous.
>>> 
>>> 
>>> 
>>> Regards,
>>> Abhimanyu
>> 

Mime
View raw message