spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: Spark streaming for synchronous API
Date Tue, 09 Sep 2014 02:41:00 GMT
Ron,

On Tue, Sep 9, 2014 at 11:27 AM, Ron's Yahoo! <zlgonzalez@yahoo.com.invalid>
 wrote:
>
>   I’m trying to figure out how I can run Spark Streaming like an API.
>   The goal is to have a synchronous REST API that runs the spark data flow
> on YARN.


I guess I *may* develop something similar in the future.

By "a synchronous REST API", do you mean that submitting the job is
synchronous and you would fetch the processing results via a different
call? Or do you want to submit a job and get the processed data back as an
HTTP stream?

To begin with, is it even possible to have Spark Streaming run as a yarn
> job?
>

I think it is very much possible to run Spark Streaming as a YARN job; at
least it worked well with Mesos.

Tobias

Mime
View raw message