spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Cheung <>
Subject RE: SparkR csv without headers
Date Fri, 21 Aug 2015 19:57:10 GMT
You could also rename them with names
Unfortunately the API doesn't show the example of that

On Thu, Aug 20, 2015 at 7:43 PM -0700, "Sun, Rui" <> wrote:

You can create a DataFrame using load.df() with a specified schema.

Something like:
schema <- structType(structField(“a”, “string”), structField(“b”, integer),
read.df ( …, schema = schema)

From: Franc Carter []
Sent: Wednesday, August 19, 2015 1:48 PM
Subject: SparkR csv without headers


Does anyone have an example of how to create a DataFrame in SparkR  which specifies the column
names - the csv files I have do not have column names in the first row. I can get read a csv
nicely with com.databricks:spark-csv_2.10:1.0.3, but I end up with column names C1, C2, C3



Franc Carter     I      Systems Architect    I     RoZetta Technology

[Description: Description: Description: cid:image003.jpg@01D02903.9B540580]

L4. 55 Harrington Street,  THE ROCKS,  NSW, 2000

PO Box H58, Australia Square, Sydney NSW, 1215, AUSTRALIA

T  +61 2 8355 2515<tel:%2B61%202%208355%202515>     I<>


DISCLAIMER: The contents of this email, inclusive of attachments, may be legally

privileged and confidential. Any unauthorised use of the contents is expressly prohibited.

View raw message