spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From andy petrella <>
Subject Re: Spark Streaming Server
Date Wed, 18 Sep 2013 19:55:21 GMT
Hello there,
I hope I get the whole thing correctly... but I think I did something quite
similar that polls the yahoo api for data, which I push into a DStream:

Any comments or even concerns (if they are constructive ^^) are more than

And, btw, I was thinking the same, this kind of use cases should be enabled
easier in the API.


On Wed, Sep 18, 2013 at 9:42 PM, Vadim Chekan <> wrote:

> Hi Paul,
> I had the same thoughts when started looking at spark.
> This link might be helpful:
> Vadim.
> On Tue, Sep 17, 2013 at 6:50 PM, Paul Snively <> wrote:
>> Hi everyone!
>> In looking at the Spark Streaming API more closely, it occurs to me that
>> it seems biased toward seeing Spark Streaming processes as consumers of
>> existing services. I'm thinking particularly of socketStream and zeroMQ
>> stream, which want a host/port and Publisher, respectively. But I think
>> this overlooks a whole class of use cases, where I'd like to establish a
>> server that someone can stream data to, and use Spark Streaming to process
>> it. Has any work been done on this front? I suppose, worst case, I could
>> write my own NetworkReceiver that listens on a socket or somesuch, but it'd
>> be nice if I didn't have to.
>> Thoughts?
>> Thanks!
>> Paul
> --
> From RFC 2631: In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT is
> explicitly specified

View raw message