kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From D Stephan <kafkastre...@gmail.com>
Subject Kafka topic inconsistency
Date Fri, 01 Dec 2017 09:07:37 GMT
Hello,

We have the following code to send avro message to a topic (Please find the
code snippet below).  Although the ProducerResord received a complete
acknowledgement withiot exception, the message is never stored in this
topic.
In the log we can see the following message is logged: "collectedReading
sent".

For your info,  we use also kafka-connect to store log in Elastic search.
In the kafka-connect log that makes connection to the related topic, we see
the following log:

[2017-11-29 23:44:19,082] ERROR
WorkerSinkTask{id=collected-interval-readings-0} Offset commit failed,
rewinding to last committed offsets
(org.apache.kafka.connect.runtime.WorkerSinkTask)

[2017-11-29 23:44:19,085] ERROR
WorkerSinkTask{id=collected-interval-readings-0} Commit of offsets threw an
unexpected exception for sequence number 78: null
(org.apache.kafka.connect.runtime.WorkerSinkTask)

We are not sure whether these connect logs were related to our issue
(message is not stored in a topic, although the kafka producerd has
received a complete acknowledgement without exception).

Do you have any ideas how to mitigate this issues?


Thanks,
Danny


Herewith is the code snippet:

Callback callback = new Callback() {
@Override
public void onCompletion(RecordMetadata metadata, Exception e) {
if (e == null) {
if (LOGGER.isInfoEnabled()) {
LOGGER.info("collectedReading sent");
}
} else {
LOGGER.error("exception sending collectedReading", e);
}
}
};


ProducerRecord<CollectedReadingKey, CollectedReading> record = new
ProducerRecord<>(topicName,
collectedReadingKey, collectedReading);

producer.send(record, callback);

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message