spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <msegel_had...@hotmail.com>
Subject Re: Spark or Storm
Date Wed, 17 Jun 2015 16:39:26 GMT
Actually the reverse.

Spark Streaming is really a micro batch system where the smallest window is 1/2 a second (500ms).

So for CEP, its not really a good idea. 

So in terms of options…. spark streaming, storm, samza, akka and others… 

Storm is probably the easiest to pick up,  spark streaming / akka may give you more flexibility
and akka would work for CEP. 

Just my $0.02

> On Jun 16, 2015, at 9:40 PM, Spark Enthusiast <sparkenthusiast@yahoo.in> wrote:
> 
> I have a use-case where a stream of Incoming events have to be aggregated and joined
to create Complex events. The aggregation will have to happen at an interval of 1 minute (or
less).
> 
> The pipeline is :
>                                   send events                                       
  enrich event
> Upstream services -------------------> KAFKA ---------> event Stream Processor
------------> Complex Event Processor ------------> Elastic Search.
> 
> From what I understand, Storm will make a very good ESP and Spark Streaming will make
a good CEP.
> 
> But, we are also evaluating Storm with Trident.
> 
> How does Spark Streaming compare with Storm with Trident?
> 
> Sridhar Chellappa
> 
> 
> 
>  
> 
> 
> 
> On Wednesday, 17 June 2015 10:02 AM, ayan guha <guha.ayan@gmail.com> wrote:
> 
> 
> I have a similar scenario where we need to bring data from kinesis to hbase. Data volecity
is 20k per 10 mins. Little manipulation of data will be required but that's regardless of
the tool so we will be writing that piece in Java pojo.
> All env is on aws. Hbase is on a long running EMR and kinesis on a separate cluster.
> TIA.
> Best
> Ayan
> On 17 Jun 2015 12:13, "Will Briggs" <wrbriggs@gmail.com <mailto:wrbriggs@gmail.com>>
wrote:
> The programming models for the two frameworks are conceptually rather different; I haven't
worked with Storm for quite some time, but based on my old experience with it, I would equate
Spark Streaming more with Storm's Trident API, rather than with the raw Bolt API. Even then,
there are significant differences, but it's a bit closer.
> 
> If you can share your use case, we might be able to provide better guidance.
> 
> Regards,
> Will
> 
> On June 16, 2015, at 9:46 PM, asoni.learn@gmail.com <mailto:asoni.learn@gmail.com>
wrote:
> 
> Hi All,
> 
> I am evaluating spark VS storm ( spark streaming  ) and i am not able to see what is
equivalent of Bolt in storm inside spark.
> 
> Any help will be appreciated on this ?
> 
> Thanks ,
> Ashish
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org <mailto:user-unsubscribe@spark.apache.org>
> For additional commands, e-mail: user-help@spark.apache.org <mailto:user-help@spark.apache.org>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org <mailto:user-unsubscribe@spark.apache.org>
> For additional commands, e-mail: user-help@spark.apache.org <mailto:user-help@spark.apache.org>
> 
> 
> 


Mime
View raw message