spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ramandeep Singh <rs5...@nyu.edu>
Subject Re: Troubleshooting Spark OOM
Date Wed, 09 Jan 2019 23:41:55 GMT
Hi,

Here are a few suggestions that you can try.

OOM Issues that, I have faced with Spark:
*Not enough shuffle partition*s.Increase them.
Less memory Overhead settings: Boosting it to around 12 percent. You
usually get this as a error message in your executors.
*Large Executor Configs*: They can be problematic, smaller and larger in
number executors are preferred over larger and fewer executors.
Changing GC algorithm

http://orastack.com/spark-scaling-to-large-datasets.html


Here are a few tips




On Wed, Jan 9, 2019 at 1:55 PM Dillon Dukek <dillon.dukek@placed.com.invalid>
wrote:

> Hi William,
>
> Just to get started, can you describe the spark version you are using and
> the language? It doesn't sound like you are using pyspark, however,
> problems arising from that can be different so I just want to be sure. As
> well, can you talk through the scenario under which you are dealing with
> this error? ie the order of operations for the transformations you are
> applying.
>
> However, if you're set on getting a heap dump, probably the easiest way
> would be to just monitor an active application through the spark UI then go
> grab a heap dump from the executor java process when you notice one that's
> having problems.
>
> On Wed, Jan 9, 2019 at 10:18 AM William Shen <willshen@marinsoftware.com>
> wrote:
>
>> Hi there,
>>
>> We've encountered Spark executor Java OOM issues for our Spark
>> application. Any tips on how to troubleshoot to identify what objects are
>> occupying the heap? In the past, dealing with JVM OOM, we've worked with
>> analyzing heap dumps, but we are having a hard time with locating Spark
>> heap dump after a crash, and we also anticipate that these heap dump will
>> be huge (since our nodes have a large memory allocation) and may be
>> difficult to analyze locally. Can someone share their experience dealing
>> with Spark OOM?
>>
>> Thanks!
>>
>

-- 
Regards,
Ramandeep Singh
Blog:http://ramannanda.blogspot.com

Mime
View raw message