spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Chu <m...@kabam.com>
Subject Re: Programatically running of the Spark Jobs.
Date Thu, 04 Sep 2014 06:45:33 GMT
https://github.com/spark-jobserver/spark-jobserver

Ooyala's Spark jobserver is the current de facto standard, IIUC. I just
added it to our prototype stack, and will begin trying it out soon. Note
that you can only do standalone or Mesos; YARN isn't quite there yet.

(The repo just moved from https://github.com/ooyala/spark-jobserver, so
don't trust Google on this one (yet); development is happening in the first
repo.)



On Wed, Sep 3, 2014 at 11:39 PM, Vicky Kak <vicky.kak@gmail.com> wrote:

> I have been able to submit the spark jobs using the submit script but I
> would like to do it via code.
> I am unable to search anything matching to my need.
> I am thinking of using org.apache.spark.deploy.SparkSubmit to do so, may
> be have to write some utility that passes the parameters required for this
> class.
> I would be interested to know how community is doing.
>
> Thanks,
> Vicky
>

Mime
View raw message