From user-return-9246-apmail-spark-user-archive=spark.apache.org@spark.apache.org Wed Jun 11 01:44:16 2014 Return-Path: X-Original-To: apmail-spark-user-archive@minotaur.apache.org Delivered-To: apmail-spark-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BCD8111B8C for ; Wed, 11 Jun 2014 01:44:16 +0000 (UTC) Received: (qmail 45493 invoked by uid 500); 11 Jun 2014 01:44:15 -0000 Delivered-To: apmail-spark-user-archive@spark.apache.org Received: (qmail 45276 invoked by uid 500); 11 Jun 2014 01:44:15 -0000 Mailing-List: contact user-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@spark.apache.org Delivered-To: mailing list user@spark.apache.org Received: (qmail 45263 invoked by uid 99); 11 Jun 2014 01:44:15 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Jun 2014 01:44:15 +0000 X-ASF-Spam-Status: No, hits=2.8 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of strebkov@gmail.com designates 209.85.219.52 as permitted sender) Received: from [209.85.219.52] (HELO mail-oa0-f52.google.com) (209.85.219.52) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Jun 2014 01:44:12 +0000 Received: by mail-oa0-f52.google.com with SMTP id j17so4860008oag.25 for ; Tue, 10 Jun 2014 18:43:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=p/IB8RCNpbSkYuaWquCytqgvlOlEtZibiNsNZt0cWnQ=; b=TeWt3+g86+A+DUjyXaFqGhgLyTIsAZYVNeEiAy992Bl+t9jpugZbDqudJFAaRCY+0f 2WtxtolSzqWbwjCrvuKZIB0AdRJzdvnBJgSSBiReq6RE0Yw4zYOJdG2N5gvJm/nld9/5 UnBfOaZo+ojdagqP0MxK9UbbTJ4gVAZv1A/Sg7AopNkms3wQoesL9LUqRVUmIVgKIVPX T8das5esa6wcmRtJda41P4lCAcMqJ3cAYAipi+M+cUjs7MrepJ5t1j9lWCUu8tAMdQeR tzE62kMiL4QN8py6ShfUisR79j7rPG4j6DylDW4B534hEXn5WpX+I/Y+u16sCVoh3Wnj LRFg== MIME-Version: 1.0 X-Received: by 10.182.199.5 with SMTP id jg5mr143279obc.75.1402451027849; Tue, 10 Jun 2014 18:43:47 -0700 (PDT) Received: by 10.76.154.233 with HTTP; Tue, 10 Jun 2014 18:43:47 -0700 (PDT) In-Reply-To: <1402450448604-7363.post@n3.nabble.com> References: <1402450448604-7363.post@n3.nabble.com> Date: Tue, 10 Jun 2014 18:43:47 -0700 Message-ID: Subject: Re: output tuples in CSV format From: Mikhail Strebkov To: user@spark.apache.org Cc: spark users group Content-Type: multipart/alternative; boundary=e89a8ff2502854776e04fb859306 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8ff2502854776e04fb859306 Content-Type: text/plain; charset=UTF-8 you can just use something like this: myRdd(_.productIterator.mkString(",")).saveAsTextFile On Tue, Jun 10, 2014 at 6:34 PM, SK wrote: > My output is a set of tuples and when I output it using saveAsTextFile, my > file looks as follows: > > (field1_tup1, field2_tup1, field3_tup1,...) > (field1_tup2, field2_tup2, field3_tup2,...) > > In Spark. is there some way I can simply have it output in CSV format as > follows (i.e. without the parentheses): > field1_tup1, field2_tup1, field3_tup1,... > field1_tup2, field2_tup2, field3_tup2,... > > I could write a script to remove the parentheses, but would be easier if I > could omit the parentheses. I did not find a saveAsCsvFile in Spark. > > thanks > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/output-tuples-in-CSV-format-tp7363.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > --e89a8ff2502854776e04fb859306 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
you can just use something like this:
=C2=A0 myRdd(_.p= roductIterator.mkString(",")).saveAsTextFile


On Tue,= Jun 10, 2014 at 6:34 PM, SK <skrishna.id@gmail.com> wro= te:
My output is a set of tuples and when I outp= ut it using saveAsTextFile, my
file looks as follows:

(field1_tup1, field2_tup1, field3_tup1,...)
(field1_tup2, field2_tup2, field3_tup2,...)

In Spark. is there some way I can simply have it output in CSV format as follows (i.e. without the parentheses):
field1_tup1, field2_tup1, field3_tup1,...
field1_tup2, field2_tup2, field3_tup2,...

I could write a script to remove the parentheses, but would be easier if I<= br> could omit the parentheses. I did not find a saveAsCsvFile in Spark.

thanks



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/output-tuples-in-CSV-= format-tp7363.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

--e89a8ff2502854776e04fb859306--