spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ji Yan <ji...@drive.ai>
Subject Re: Spark/Mesos with GPU support
Date Fri, 30 Dec 2016 19:02:16 GMT
Thanks Michael, Tim and I have touched base and thankfully the issue has
already been resolved

On Fri, Dec 30, 2016 at 9:20 AM, Michael Gummelt <mgummelt@mesosphere.io>
wrote:

> I've cc'd Tim and Kevin, who worked on GPU support.
>
> On Wed, Dec 28, 2016 at 11:22 AM, Ji Yan <jiyan@drive.ai> wrote:
>
>> Dear Spark Users,
>>
>> Has anyone had successful experience running Spark on Mesos with GPU
>> support? We have a Mesos cluster that can see and offer nvidia GPU
>> resources. With Spark, it seems that the GPU support with Mesos (
>> https://github.com/apache/spark/pull/14644) has only recently been
>> merged into Spark Master which is not found in 2.0.2 release yet. We have a
>> custom built Spark from 2.1-rc5 which is confirmed to have the above
>> change. However when we try to run any code from Spark on this Mesos setup,
>> the spark program hangs and keeps saying
>>
>> “WARN TaskSchedulerImpl: Initial job has not accepted any resources;
>> check your cluster UI to ensure that workers are registered and have
>> sufficient resources”
>>
>> We are pretty sure that the cluster has enough resources as there is
>> nothing running on it. If we disable the GPU support in configuration and
>> restart mesos and retry the same program, it would work.
>>
>> Any comment/advice on this greatly appreciated
>>
>> Thanks,
>> Ji
>>
>>
>> The information in this email is confidential and may be legally
>> privileged. It is intended solely for the addressee. Access to this email
>> by anyone else is unauthorized. If you are not the intended recipient, any
>> disclosure, copying, distribution or any action taken or omitted to be
>> taken in reliance on it, is prohibited and may be unlawful.
>>
>
>
>
> --
> Michael Gummelt
> Software Engineer
> Mesosphere
>

-- 
 

The information in this email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this email 
by anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be 
taken in reliance on it, is prohibited and may be unlawful.

Mime
View raw message