kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bo Xu <box...@gmail.com>
Subject Re: messages intermittently dropped in a single partition configuration
Date Thu, 28 Apr 2016 15:31:12 GMT
PS: The message dropping occurred intermittently, not all at the end. For
example, it is the 10th, 15th, 18th messages that are missing. It it were
all at the end, it would be understandable because I'm not using flush() to
force transmitting.


On Thu, Apr 28, 2016 at 10:15 AM, Bo Xu <boxu08@gmail.com> wrote:

> I set up a simple Kafka configuration, with one topic and one partition. I
> have a Python producer to continuously publish messages to the Kafka server
> and a Python consumer to receive messages from the server. Each message is
> about 10K bytes, far smaller than socket.request.max.bytes=104857600. What
> I found is that the consumer intermittently missed some messages. I checked
> the server log file and found that these messages are missing there as
> well. So it looks like these message were never stored by the server. I
> also made sure that the producer did not receive any error for every
> message that it published (using send()).
> Any clues what could have caused the problem?
> Thanks,
> Bo

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message