kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Compton <daniel.compton.li...@gmail.com>
Subject Re: Cache Memory Kafka Process
Date Mon, 27 Jul 2015 21:23:33 GMT
http://www.linuxatemyram.com may be a helpful resource to explain this
better.
On Tue, 28 Jul 2015 at 5:32 AM Ewen Cheslack-Postava <ewen@confluent.io>
wrote:

> Having the OS cache the data in Kafka's log files is useful since it means
> that data doesn't need to be read back from disk when consumed. This is
> good for the latency and throughput of consumers. Usually this caching
> works out pretty well, keeping the latest data from your topics in cache
> and only pulling older data into memory if a consumer reads data from
> earlier in the log. In other words, by leveraging OS-level caching of
> files, Kafka gets an in-memory caching layer for free.
>
> Generally you shouldn't need to clear this data -- the OS should only be
> using memory that isn't being used anyway. Is there a particular problem
> you're encountering that clearing the cache would help with?
>
> -Ewen
>
> On Mon, Jul 27, 2015 at 2:33 AM, Nilesh Chhapru <
> nilesh.chhapru@ugamsolutions.com> wrote:
>
> > Hi All,
> >
> > I am facing issues with kafka broker process taking  a lot of cache
> > memory, just wanted to know if the process really need that much of
> > cache memory, or can i clear the OS level cache by setting a cron.
> >
> > Regards,
> > Nilesh Chhapru.
> >
>
>
>
> --
> Thanks,
> Ewen
>
-- 
--
Daniel

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message