kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: Understanding java options used by Kafka
Date Mon, 23 Sep 2013 16:01:27 GMT
I'm not sure about the reason for -Djava.awt.headless=true, but the GC
options were picked after quite a lot of GC tuning for the client/server
configuration that LinkedIn has deployed in production. This works well
when producers are deployed with request.required.acks=1 and consumers with
fetch.wait.max.ms set to default value (100 ms). The garbage collection
settings are optimized for preventing objects from getting promoted to old
generation. An option that helped in reducing the promotion rate is
MaxTenuringThreshold, which is set to max (15 or 31 depending on your JVM).
We found that sizing the young generation and survivor spaces small enough
with a small heap worked better than having a large young gen and large
heap, in terms of the max GC pause.

Hope that helps.

Thanks,
Neha


On Fri, Sep 20, 2013 at 12:40 PM, Marc Labbe <mrlabbe@gmail.com> wrote:

> Hi,
>
> I am fairly new to java and I am trying to figure out the reason behind
> each option listed on http://kafka.apache.org/documentation.html#java.
> Sorry if this a noob question :)
>
> In this list, one of option particularly struck my attention and had me
> wondering if the whole list of options is correct:
>
> -Djava.awt.headless=true
>
> My understanding is the AWT is a toolkit for UI applications, which is not
> really the case of Kafka. Is this correct?
>
> Also, is it possible to have more information on why some specific GC
> options are used at LinkedIn? Is there anything in there specifically
> helping Kafka's performance?
>
> I am thinking of NewSize, MaxNewSixze, UseConcMarkSweepGC and
> CMSInitiatingOccupancyFraction.
>
> I looked them up and found what they are but is there an explanation about
> why/if we should use them with Kafka?
>
> thanks
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message