spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim St Clair <tstcl...@redhat.com>
Subject Re: is Mesos falling out of favor?
Date Fri, 09 May 2014 13:46:46 GMT




----- Original Message -----
> From: "deric" <barton.tomas@gmail.com>
> To: user@spark.incubator.apache.org
> Sent: Tuesday, May 6, 2014 11:42:58 AM
> Subject: Re: is Mesos falling out of favor?

Nope.

> 
> I guess it's due to missing documentation and quite complicated setup.
> Continuous integration would be nice!

The setup is pretty simple, but stack integration tests are certainly missing, and Spark POM's
have been out of date for some time.  There are JIRAs in both projects to clean up integration
and update. 

> 
> Btw. is it possible to use spark as a shared library and not to fetch spark
> tarball for each task?

It's really easy to edit Spark's-Mesos scheduler and executor to do what you want.  e.g.
run local binaries, etc.

> 
> Do you point SPARK_EXECUTOR_URI to HDFS url?

Typically yes, but again it's pretty easy edit to the scheduler and executor to do what you
want. 

> 
> 
> 
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/is-Mesos-falling-out-of-favor-tp5444p5448.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
> 

-- 
Cheers,
Tim
Freedom, Features, Friends, First -> Fedora
https://fedoraproject.org/wiki/SIGs/bigdata

Mime
View raw message