spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guy Doulberg <Guy.Doulb...@perion.com>
Subject RE: Jetty as spark straming input
Date Thu, 27 Nov 2014 07:50:55 GMT
Thanks rektide. 



-----Original Message-----
From: rektide@voodoowarez.com [mailto:rektide@voodoowarez.com] 
Sent: Wednesday, November 26, 2014 8:43 PM
To: Guy Doulberg
Cc: user@spark.apache.org
Subject: Re: Jetty as spark straming input

On Wed, Nov 26, 2014 at 04:06:40PM +0000, Guy Doulberg wrote:
> Hi guys
> 
> I started playing with spark streaming, and I came up with an idea that I wonder if it
a  valid idea.
> 
> Building a jetty input stream, which is basically a jetty server that each http request
it gets it streams .
> 
> What do you think of this idea?

If Spark Streaming could get closer to realtime there would be more interesting prospects
here. But as it is, in most use cases the discritization window is going to mean more latency
than you want to have for handling requests.

There are still some interesting analytics use cases in pushing http traffic into Spark Streaming.
Some process-heavy API systems might be able to do full request/response handling chiefly
via Spark Streaming. There is a reciprocal quesiton fo how to get data back to Jetty.

Mongrel2 and Ilya's article "Building a Modern [front-end routed] Web Stack" both come to
mind- http://mongrel2.org/manual/book-finalch2.html
https://www.igvita.com/2012/01/18/building-a-modern-web-stack-for-the-realtime-web/

To really get deeply into the Realtime web-stack, we'd need to see Spark Streaming emerge
some closer-to-realtime/immediate-handling methods.

-rektide

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message