spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil <ak...@sigmoidanalytics.com>
Subject Re: Spark Streaming Applications
Date Tue, 28 Oct 2014 06:55:26 GMT
You can check  this project out
<https://github.com/sigmoidanalytics/spork-streaming/>   (it is a bit
outdated, but works) It is basically the integration of Pig on
SparkStreaming. You can write pig scripts and they are underneath executed
as spark streaming job. To get you started quickly,  have a look at the 
SparkStreamingLauncher.java
<https://github.com/sigmoidanalytics/spork-streaming/blob/master/src/org/apache/pig/backend/hadoop/executionengine/spark_streaming/SparkStreamingLauncher.java>
 
Where it actually  calls the streaming
<https://github.com/sigmoidanalytics/spork-streaming/blob/master/src/org/apache/pig/backend/hadoop/executionengine/spark_streaming/SparkStreamingLauncher.java#L183>
 
and all, this is a pretty big project actually.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Applications-tp16976p17440.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message