spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Grishchenko <programme...@gmail.com>
Subject Re: Help Explain Tasks in WebUI:4040
Date Fri, 28 Aug 2015 22:38:09 GMT
It really depends on the code. I would say that the easiest way is to
restart the problematic action, find the straggler task and analyze whats
happening with it with jstack / make a heap dump and analyze locally. For
example, there might be the case that your tasks are connecting to some
external resource and this resource is timing out under the pressure. Also
call toDebugString on the problematic RDD before calling an action that
triggers the calculations, this would give you an understanding what your
execution tasks are really doing

On Fri, Aug 28, 2015 at 7:47 PM, Muler <mulugeta.abebaw@gmail.com> wrote:

> I have a 7 node cluster running in standalone mode (1 executor per node,
> 100g/executor, 18 cores/executor)
>
> Attached is the Task status for two of my nodes. I'm not clear why some of
> my tasks are taking too long:
>
>    1. [node sk5, green] task 197 took 35 mins while task 218 took less
>    than 2 mins. But if you look into the size of output size/records they have
>    almost same size. Even more strange, the size of shuffle spill for memory
>    and disk is 0 for task 197 and yet it is taking a long time
>
> Same issue for my other node (sk3, red)
>
> Can you please explain what is going on?
>
> Thanks,
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>



-- 
Alexey Grishchenko, http://0x0fff.com

Mime
View raw message