spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "@Sanjiv Singh" <sanjiv.is...@gmail.com>
Subject Re: How to convert a non-rdd data to rdd.
Date Sun, 12 Oct 2014 07:28:29 GMT
Hi Karthik,

Can you provide us more detail of dataset "data" that you wanted to
parallelize with

SparkContext.parallelize(data);




Regards,
Sanjiv Singh


Regards
Sanjiv Singh
Mob :  +091 9990-447-339

On Sun, Oct 12, 2014 at 11:45 AM, rapelly kartheek <kartheek.mbms@gmail.com>
wrote:

> Hi,
>
> I am trying to write a String that is not an rdd to HDFS. This data is a
> variable in Spark Scheduler code. None of the spark File operations are
> working because my data is not rdd.
>
> So, I tried using SparkContext.parallelize(data). But it throws error:
>
> [error]
> /home/karthik/spark-1.0.0/core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala:265:
> not found: value SparkContext
> [error]      SparkContext.parallelize(result)
> [error]      ^
> [error] one error found
>
> I realized that this data is part of the Scheduler. So, the Sparkcontext
> would not have got created yet.
>
> Any help in "writing scheduler variable data to HDFS" is appreciated!!
>
> -Karthik
>

Mime
View raw message