Hi Karthik,

Can you provide us more detail of dataset "data" that you wanted to parallelize with 


Sanjiv Singh

Sanjiv Singh
Mob :  +091 9990-447-339

On Sun, Oct 12, 2014 at 11:45 AM, rapelly kartheek <kartheek.mbms@gmail.com> wrote:

I am trying to write a String that is not an rdd to HDFS. This data is a variable in Spark Scheduler code. None of the spark File operations are working because my data is not rdd. 

So, I tried using SparkContext.parallelize(data). But it throws error:

[error] /home/karthik/spark-1.0.0/core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala:265: not found: value SparkContext
[error]      SparkContext.parallelize(result)
[error]      ^
[error] one error found

I realized that this data is part of the Scheduler. So, the Sparkcontext would not have got created yet.

Any help in "writing scheduler variable data to HDFS" is appreciated!!