spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akash Mishra <akash.mishr...@gmail.com>
Subject Re: Spark Job Stuck in on stage.
Date Wed, 26 Aug 2015 15:29:03 GMT
PFA executor List.

[image: Inline image 2]


On Wed, Aug 26, 2015 at 4:37 PM, Sameer Farooqui <sameerf@databricks.com>
wrote:

> Hmm... how much memory does each Executor JVM have? And how many total
> Executors are there? Also is the task you're running very computationally
> expensive?
>
> A task should ideally run for like 100 ms - 10 seconds or so. Hours for
> each task looks off to me. Also, why did some tasks finish in 8 hours,
> which others are running for 18th?
>
>
>
> On Wed, Aug 26, 2015 at 2:37 PM, Akash Mishra <akash.mishra20@gmail.com>
> wrote:
>
>> I am currently running a Spark job on Mesos. On Stages UI page, I see
>> that *GC Time* is too much for each task. 16.2hr in 18.2hr Duration.
>>
>> Please see attached pic
>>
>> [image: Inline image 1]
>>
>>
>> Does this means that there are lots of Garbage Collection with are
>> happening and hence the job are stuck and not able to complete.
>>
>> Thanks,
>> --
>>
>> Regards,
>> Akash Mishra.
>>
>>
>> "Its not our abilities that make us, but our decisions."--Albus Dumbledore
>>
>
>


-- 

Regards,
Akash Mishra.


"Its not our abilities that make us, but our decisions."--Albus Dumbledore

Mime
View raw message