spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kundan kumar <iitr.kun...@gmail.com>
Subject Writing RDD to a csv file
Date Tue, 03 Feb 2015 13:09:36 GMT
I have a RDD which is of type

org.apache.spark.rdd.RDD[(String, (Array[String], Option[Array[String]]))]

I want to write it as a csv file.

Please suggest how this can be done.

myrdd.map(line => (line._1 + "," + line._2._1.mkString(",") + "," +
line._2._2.mkString(','))).saveAsTextFile("hdfs://...")

Doing mkString on line._2._1 works but does not work for the Option type.

Please suggest how this can be done.


Thanks
Kundan

Mime
View raw message