spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: G1 GC takes too much time
Date Mon, 30 May 2016 01:47:05 GMT
bq. It happens during the Reduce majority.

Did the above refer to reduce operation ?

Can you share your G1GC parameters (and heap size for workers) ?

Thanks

On Sun, May 29, 2016 at 6:15 PM, condor join <spark_kernal@outlook.com>
wrote:

> Hi,
> my spark application failed due to take too much time during GC. Looking
> at the logs I found these things:
> 1.there are Young GC takes too much time,and not found Full GC happen this;
> 2.The time takes too much during the object copy;
> 3.It happened  more easily when there were not enough resources;
> 4.It happens during the Reduce majority.
>
> have anyone met the same question?
> thanks
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>

Mime
View raw message