kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rammohan Vanteru <ramz.moha...@gmail.com>
Subject Re: Kafka memory estimation
Date Sat, 13 Apr 2019 03:20:14 GMT
Hi Steve,

We are using Prometheus jmx exporter and Prometheus to scrape metrics based
on memory metric we are measuring.

Jmx exporter:
https://github.com/prometheus/jmx_exporter/blob/master/README.md

Thanks,
Ramm.

On Fri, Apr 12, 2019 at 12:43 PM Steve Howard <steve.howard@confluent.io>
wrote:

> Hi Rammohan,
>
> How are you measuring "Kafka seems to be reserving most of the memory"?
>
> Thanks,
>
> Steve
>
> On Thu, Apr 11, 2019 at 11:53 PM Rammohan Vanteru <ramz.mohan88@gmail.com>
> wrote:
>
> > Hi Users,
> >
> >  As per the article here:
> > https://docs.confluent.io/current/kafka/deployment.html#memory memory
> > requirement is roughly calculated based on formula: write throughput*30
> > (buffer time in seconds), which fits in our experiment i.e. 30MB/s*30~
> > 900MB. Followup questions here:
> >
> >    - How do we estimate the figure ’30 seconds’ for our buffer time?
> >    - Kafka seems to be reserving most of the memory even when the none of
> >    the messages are being streamed.. how does that play out.
> >
> > Any information provided would be helpful.
> >
> > Thanks,
> > Rammohan.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message