kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shivam Sharma <28shivamsha...@gmail.com>
Subject Re: Need broker side property which can restrict a fetch.size
Date Fri, 12 Jan 2018 08:56:26 GMT
Thanks Srinath, We will try to do quota management.

On Wed, Jan 10, 2018 at 1:18 PM, Srinath C <srinath.c@gmail.com> wrote:

> Hi Shivam,
>
> You can configure quotas per user/client.
>
> [1] https://kafka.apache.org/documentation/#design_quotas
>
> Regards,
> Srinath
>
> On Jan 9, 2018 5:19 PM, "Shivam Sharma" <28shivamsharma@gmail.com> wrote:
>
> > Hi,
> >
> > We are having Kafka cluster(0.11.0.0) of 3 Machines. When we increase
> > *max.fetch.size.per.partition* to around 50 MB for topic having 48
> > partition, our Kafka Broker is getting OOM.
> >
> > As our cluster has many consumers, so restricting fetch size on consumer
> > side is very difficult which is continuously leading to OOM like issues
> as
> > consumer has power to do this.
> >
> > Anything can be done on broker side to avoid these issues by restricting
> > clients.
> >
> > Thanks
> >
> > --
> > Shivam Sharma
> > Data Engineer @ Goibibo
> > Indian Institute Of Information Technology, Design and Manufacturing
> > Jabalpur
> > Mobile No- (+91) 8882114744
> > Email:- 28shivamsharma@gmail.com
> > LinkedIn:-*https://www.linkedin.com/in/28shivamsharma
> > <https://www.linkedin.com/in/28shivamsharma>*
> >
>



-- 
Shivam Sharma
Data Engineer @ Goibibo
Indian Institute Of Information Technology, Design and Manufacturing
Jabalpur
Mobile No- (+91) 8882114744
Email:- 28shivamsharma@gmail.com
LinkedIn:-*https://www.linkedin.com/in/28shivamsharma
<https://www.linkedin.com/in/28shivamsharma>*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message