spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shao, Saisai" <>
Subject RE: output tuples in CSV format
Date Wed, 11 Jun 2014 01:44:03 GMT
It would be better to add one more transformation step before saveAsTextFile, like: => "%s,%s,%s".format(tuple._1, tuple._2, tuple._3)).saveAsTextFile(...)

By manually convert to the format you what, and then write to HDFS.


-----Original Message-----
From: SK [] 
Sent: Wednesday, June 11, 2014 9:34 AM
Subject: output tuples in CSV format

My output is a set of tuples and when I output it using saveAsTextFile, my file looks as follows:

(field1_tup1, field2_tup1, field3_tup1,...) (field1_tup2, field2_tup2, field3_tup2,...)

In Spark. is there some way I can simply have it output in CSV format as follows (i.e. without
the parentheses):
field1_tup1, field2_tup1, field3_tup1,...
field1_tup2, field2_tup2, field3_tup2,...

I could write a script to remove the parentheses, but would be easier if I could omit the
parentheses. I did not find a saveAsCsvFile in Spark.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message