spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akshay Bhardwaj <akshay.bhardwaj1...@gmail.com>
Subject Re: Spark job gets hung on cloudera cluster
Date Thu, 16 May 2019 13:49:40 GMT
One of the reason that any jobs running on YARN (Spark, MR, Hive, etc) can
get stuck is if there is data unavailability issue with HDFS.
This can arise if either the Namenode is not reachable or if the particular
data block is unavailable due to node failures.

Can you check if your YARN service can communicate with Name node service?

Akshay Bhardwaj
+91-97111-33849


On Thu, May 16, 2019 at 4:27 PM Rishi Shah <rishishah.star@gmail.com> wrote:

> on yarn
>
> On Thu, May 16, 2019 at 1:36 AM Akshay Bhardwaj <
> akshay.bhardwaj1988@gmail.com> wrote:
>
>> Hi Rishi,
>>
>> Are you running spark on YARN or spark's master-slave cluster?
>>
>> Akshay Bhardwaj
>> +91-97111-33849
>>
>>
>> On Thu, May 16, 2019 at 7:15 AM Rishi Shah <rishishah.star@gmail.com>
>> wrote:
>>
>>> Any one please?
>>>
>>> On Tue, May 14, 2019 at 11:51 PM Rishi Shah <rishishah.star@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> At times when there's a data node failure, running spark job doesn't
>>>> fail - it gets stuck and doesn't return. Any setting can help here? I would
>>>> ideally like to get the job terminated or executors running on those data
>>>> nodes fail...
>>>>
>>>> --
>>>> Regards,
>>>>
>>>> Rishi Shah
>>>>
>>>
>>>
>>> --
>>> Regards,
>>>
>>> Rishi Shah
>>>
>>
>
> --
> Regards,
>
> Rishi Shah
>

Mime
View raw message