Hi Paul,

I had the same thoughts when started looking at spark.
This link might be helpful: https://github.com/mesos/spark/blob/master/docs/streaming-custom-receivers.md

Vadim.



On Tue, Sep 17, 2013 at 6:50 PM, Paul Snively <psnively@icloud.com> wrote:
Hi everyone!

In looking at the Spark Streaming API more closely, it occurs to me that it seems biased toward seeing Spark Streaming processes as consumers of existing services. I'm thinking particularly of socketStream and zeroMQ stream, which want a host/port and Publisher, respectively. But I think this overlooks a whole class of use cases, where I'd like to establish a server that someone can stream data to, and use Spark Streaming to process it. Has any work been done on this front? I suppose, worst case, I could write my own NetworkReceiver that listens on a socket or somesuch, but it'd be nice if I didn't have to.

Thoughts?

Thanks!
Paul




--
From RFC 2631: In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT is explicitly specified