spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <>
Subject Re: Spark streaming for synchronous API
Date Tue, 09 Sep 2014 02:41:00 GMT

On Tue, Sep 9, 2014 at 11:27 AM, Ron's Yahoo! <>
>   I’m trying to figure out how I can run Spark Streaming like an API.
>   The goal is to have a synchronous REST API that runs the spark data flow
> on YARN.

I guess I *may* develop something similar in the future.

By "a synchronous REST API", do you mean that submitting the job is
synchronous and you would fetch the processing results via a different
call? Or do you want to submit a job and get the processed data back as an
HTTP stream?

To begin with, is it even possible to have Spark Streaming run as a yarn
> job?

I think it is very much possible to run Spark Streaming as a YARN job; at
least it worked well with Mesos.


View raw message