spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: Driver OOM after upgrading to 1.5
Date Wed, 09 Sep 2015 21:31:26 GMT
It's likely that with codegen, you need a bigger permgen space. Also I
found that Java 7 doesn't do very well w.r.t. flushing code cache. As a
result, Spark SQL and DataFrames now run much better under Java 8, because
it flushes code cache better.


On Wed, Sep 9, 2015 at 2:12 PM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:

> Java 7.
>
> FWIW I was just able to get it to work by increasing MaxPermSize to 256m.
>
> -Sandy
>
> On Wed, Sep 9, 2015 at 11:37 AM, Reynold Xin <rxin@databricks.com> wrote:
>
>> Java 7 / 8?
>>
>> On Wed, Sep 9, 2015 at 10:10 AM, Sandy Ryza <sandy.ryza@cloudera.com>
>> wrote:
>>
>>> I just upgraded the spark-timeseries
>>> <https://github.com/cloudera/spark-timeseries> project to run on top of
>>> 1.5, and I'm noticing that tests are failing with OOMEs.
>>>
>>> I ran a jmap -histo on the process and discovered the top heap items to
>>> be:
>>>    1:        163428       22236064  <methodKlass>
>>>    2:        163428       21112648  <constMethodKlass>
>>>    3:         12638       14459192  <constantPoolKlass>
>>>    4:         12638       13455904  <instanceKlassKlass>
>>>    5:         10539        7642528  <constantPoolCacheKlass>
>>>
>>> Not sure whether this is suspicious.  Any ideas?
>>>
>>> -Sandy
>>>
>>
>>
>

Mime
View raw message