kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Quinlivan <rquinli...@signal.co>
Subject Messages rejected by broker due to size
Date Wed, 01 Mar 2017 17:33:50 GMT

I'm looking for a bit of clarification on the outcome of a
RecordTooLargeException in the broker that was not evident from the

First, what happens to a large record that is rejected? Is it ignored by
the broker or is there a way to recover a rejected message?

Second, is the message size condition checked before or after
compression/batching is applied? What exactly is "message.max.bytes"
looking at?

Third, what happens if a broker accepts the record from the producer but
the "replica.fetch.max.bytes" setting is lower than the "message.max.bytes"
setting? This seems like an edge case to me. The leader would accept the
record but replicas would not be able to receive it, so it would be lost.
Or does the replica take the max of those two settings in order to avoid
this condition?

Thanks in advance!
Robert Quinlivan
Software Engineer, Signal

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message