kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Agostino Calamita <agostino.calam...@gmail.com>
Subject Lost message in asynchronous kafka producer
Date Fri, 16 Sep 2016 08:25:16 GMT
Hi,
I have 2 brokers with a topic with replication factor = 2.
Brokers are configured with min.insync.replicas=2.

I use a producer in asynchronous mode to send 100000 messages.

After some seconds after producer start, I stop one broker.

On producer side I got no exceptions, so for producer application is all
OK, but in broker log message I see
 ERROR [Replica Manager on Broker 0]: Error processing append operation on
partition diameternew-10 (kafka.server.ReplicaManager)

So I lost all messages after broker shutdown.

Is there a way to catch as soon as possible this kind of exeption ?

I know that I can use

     m_kafkaProducer.send(prMessage).get();

or

    Future _future = m_kafkaProducer.send(prMessage);

or callback in send method, but they are very slow.


Thank.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message