spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rachana Srivastava <Rachana.Srivast...@markmonitor.com>
Subject Can Spark Provide Multiple Context Support?
Date Tue, 08 Sep 2015 17:12:29 GMT
Question: How does Spark support multiple context?

Background:  I have a stream of data coming to Spark from Kafka.   For each data in the stream
I want to download some files from HDFS and process the file data.  I have written code to
process the file from HDFS and I have code written to process stream data from Kafka using
SparkStreaming API.  I have not been able to link both.

Can you please let me know if it is feasible to create JavaRDD from file inside SparkStreamingRDD
job processing step?

Thanks,

Rachana

Mime
View raw message