spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vicky Kak <vicky....@gmail.com>
Subject Re: Programatically running of the Spark Jobs.
Date Thu, 04 Sep 2014 13:01:20 GMT
I don't think so.


On Thu, Sep 4, 2014 at 5:36 PM, Ruebenacker, Oliver A <
Oliver.Ruebenacker@altisource.com> wrote:

>
>
>      Hello,
>
>
>
>   Can this be used as a library from within another application?
>
>   Thanks!
>
>
>
>      Best, Oliver
>
>
>
> *From:* Matt Chu [mailto:mchu@kabam.com]
> *Sent:* Thursday, September 04, 2014 2:46 AM
> *To:* Vicky Kak
> *Cc:* user
> *Subject:* Re: Programatically running of the Spark Jobs.
>
>
>
> https://github.com/spark-jobserver/spark-jobserver
>
>
>
> Ooyala's Spark jobserver is the current de facto standard, IIUC. I just
> added it to our prototype stack, and will begin trying it out soon. Note
> that you can only do standalone or Mesos; YARN isn't quite there yet.
>
>
>
> (The repo just moved from https://github.com/ooyala/spark-jobserver, so
> don't trust Google on this one (yet); development is happening in the first
> repo.)
>
>
>
>
>
> On Wed, Sep 3, 2014 at 11:39 PM, Vicky Kak <vicky.kak@gmail.com> wrote:
>
> I have been able to submit the spark jobs using the submit script but I
> would like to do it via code.
>
> I am unable to search anything matching to my need.
>
> I am thinking of using org.apache.spark.deploy.SparkSubmit to do so, may
> be have to write some utility that passes the parameters required for this
> class.
>
> I would be interested to know how community is doing.
>
> Thanks,
> Vicky
>
>
>
>
> ***********************************************************************************************************************
>
> This email message and any attachments are intended solely for the use of
> the addressee. If you are not the intended recipient, you are prohibited
> from reading, disclosing, reproducing, distributing, disseminating or
> otherwise using this transmission. If you have received this message in
> error, please promptly notify the sender by reply email and immediately
> delete this message from your system.
> This message and any attachments may contain information that is
> confidential, privileged or exempt from disclosure. Delivery of this
> message to any person other than the intended recipient is not intended to
> waive any right or privilege. Message transmission is not guaranteed to be
> secure or free of software viruses.
>
> ***********************************************************************************************************************
>

Mime
View raw message