spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gerard Maas <gerard.m...@gmail.com>
Subject Re: RDD for Storm Streaming in Spark
Date Tue, 23 Dec 2014 10:17:40 GMT
I'm not aware of a project trying to achieve this integration. At some
point Summingbird had the intention of adding an Spark port, and that could
potentially bridge Storm and Spark. Not sure if that evolved into something
concrete.

In any case, an attempt to bring Storm and Spark together will consist  in
Storm collecting the data to construct an RDD and provide that to Spark for
processing, basically what Spark Streaming is doing.

-kr, Gerard.



On Tue, Dec 23, 2014 at 9:49 AM, Ajay <ajay.garga@gmail.com> wrote:

> Hi,
>
> The question is to do streaming in Spark with Storm (not using Spark
> Streaming).
>
> The idea is to use Spark as a in-memory computation engine and static data
> coming from Cassandra/Hbase and streaming data from Storm.
>
> Thanks
> Ajay
>
>
> On Tue, Dec 23, 2014 at 2:03 PM, Gerard Maas <gerard.maas@gmail.com>
> wrote:
>
>> Hi,
>>
>> I'm not sure what you are asking:
>>
>> Whether we can use spouts and bolts in Spark  (=> no)
>>
>> or whether we can do streaming in Spark:
>> http://spark.apache.org/docs/latest/streaming-programming-guide.html
>>
>> -kr, Gerard.
>>
>>
>> On Tue, Dec 23, 2014 at 9:03 AM, Ajay <ajay.garga@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Can we use Storm Streaming as RDD in Spark? Or any way to get Spark work
>>> with Storm?
>>>
>>> Thanks
>>> Ajay
>>>
>>
>>
>

Mime
View raw message