spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayur Rustagi <mayur.rust...@gmail.com>
Subject Re: Run spark on mesos remotely
Date Thu, 27 Mar 2014 20:09:45 GMT
Yes but you have to maintain connection of that machine to the master
cluster as the driver with DAG scheduler runs on that machine.
Regards
Mayur

Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>



On Thu, Mar 27, 2014 at 4:09 AM, Wush Wu <wush@bridgewell.com> wrote:

> Dear all,
>
> We have a spark 0.8.1 cluster on mesos 0.15. It works if I submit the job
> from the master of mesos. That is to say, I spawn the spark shell or launch
> the scala application on the master of mesos.
>
> However, when I submit the job from another machine, the job will lost.
> The logs shows that the mesos does not copy the
> spark-0.8.1-incubating.tar.gz to the temporal working directory, so the job
> lost immediately. Is it possible to submit the job from the machine not
> belong to mesos cluster?
>
> Thanks!
>
> Wush
>

Mime
View raw message