kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaikiran Pai <jai.forums2...@gmail.com>
Subject Re: Out of memory - Java Heap space
Date Wed, 27 Apr 2016 09:02:00 GMT
Have you tried getting the memory usage output using tool like jmap and 
seeing what's consuming the memory? Also, what are you heap sizes for 
the process?

-Jaikiran

On Tuesday 19 April 2016 02:31 AM, McKoy, Nick wrote:
> To follow up with my last email, I have been looking into socket.receive.buffer.byte
as well as socket.send.buffer.bytes. Would it help to increase the buffer for OOM issue?
>
> All help is appreciated!
>
> Thanks!
>
>
> -nick
>
>
> From: "McKoy, Nick" <Nicholas.McKoy@washpost.com<mailto:Nicholas.McKoy@washpost.com>>
> Date: Monday, April 18, 2016 at 3:41 PM
> To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" <users@kafka.apache.org<mailto:users@kafka.apache.org>>
> Subject: Out of memory - Java Heap space
>
> Hey all,
>
> I have a kafka cluster of 5 nodes that’s working really hard. CPU is around 40% idle
daily.
>
> I looked at the file descriptor note on this documentation page http://docs.confluent.io/1.0/kafka/deployment.html#file-descriptors-and-mmap
and decided to give it a shot on one instance in the cluster just to see how it performed.
I increased this number to 1048576.
>
> I kept getting this error from the kafka logs:
> ERROR [ReplicaFetcherThread--1-6], Error due to (kafka.server.ReplicaFetcherThread) java.lang.OutOfMemoryError:
Java heap space
>
> I increased heap to see if that would help and I kept seeing these errors. Could the
file descriptor change have something related to this?
>
>
>
> —
> Nicholas McKoy
> Engineering – Big Data and Personalization
> Washington Post Media
>
> One Franklin Square, Washington, DC 20001
> Email: nicholas.mckoy@washpost.com<mailto:nicholas.mckoy@washpost.com>
>


Mime
View raw message