kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Satish Duggana <satish.dugg...@gmail.com>
Subject Re: Determining broker max buffer size from producer.
Date Mon, 03 Sep 2018 10:03:32 GMT
Are you asking about the maximum message size that can be sent from a
producer?
max.message.bytes is used to control that at broker/topic level. You can
find more details
at the below links.

*"The largest record batch size allowed by Kafka. If this is increased and
there are consumers older than 0.10.2, the consumers' fetch size must also
be increased so that the they can fetch record batches this large. In the
latest message format version, records are always grouped into batches for
efficiency. In previous message format versions, uncompressed records are
not grouped into batches and this limit only applies to a single record in
that case."*


https://kafka.apache.org/documentation/#topicconfigs
https://kafka.apache.org/documentation/#brokerconfigs

Thanks,
Satish.

On Mon, Sep 3, 2018 at 2:12 PM, claude.warren@wipro.com <
claude.warren@wipro.com> wrote:

> Greetings,
>
>
> I have spent several hours looking through documentation and historical
> email chains and have been unable to find a solution to my problem.
>
>
> I have a case where I need to construct a producer and I need to know the
> broker max buffer size so that I don't get any nasty buffer too big errors
> from the server when serializing large objects.  I have been unable to find
> any way to do this short of reading the broker configuration and I can't do
> that from a remote call (as far as I can tell).  Does anybody have a
> solution?
>
>
> Claude
>
> The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus transmitted by this email.
> www.wipro.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message