spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rapelly kartheek <kartheek.m...@gmail.com>
Subject How to convert a non-rdd data to rdd.
Date Sun, 12 Oct 2014 06:15:08 GMT
Hi,

I am trying to write a String that is not an rdd to HDFS. This data is a
variable in Spark Scheduler code. None of the spark File operations are
working because my data is not rdd.

So, I tried using SparkContext.parallelize(data). But it throws error:

[error]
/home/karthik/spark-1.0.0/core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala:265:
not found: value SparkContext
[error]      SparkContext.parallelize(result)
[error]      ^
[error] one error found

I realized that this data is part of the Scheduler. So, the Sparkcontext
would not have got created yet.

Any help in "writing scheduler variable data to HDFS" is appreciated!!

-Karthik

Mime
View raw message