spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiangrui Meng <men...@gmail.com>
Subject Re: WARN from Similarity Calculation
Date Tue, 17 Feb 2015 21:58:51 GMT
It may be caused by GC pause. Did you check the GC time in the Spark
UI? -Xiangrui

On Sun, Feb 15, 2015 at 8:10 PM, Debasish Das <debasish.das83@gmail.com> wrote:
> Hi,
>
> I am sometimes getting WARN from running Similarity calculation:
>
> 15/02/15 23:07:55 WARN BlockManagerMasterActor: Removing BlockManager
> BlockManagerId(7, abc.com, 48419, 0) with no recent heart beats: 66435ms
> exceeds 45000ms
>
> Do I need to increase the default 45 s to larger values for cases where we
> are doing blocked operation or long compute in the mapPartitions ?
>
> Thanks.
> Deb

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message