spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sun, Rui" <rui....@intel.com>
Subject RE: SparkR csv without headers
Date Fri, 21 Aug 2015 02:43:12 GMT
Hi,

You can create a DataFrame using load.df() with a specified schema.

Something like:
schema <- structType(structField(“a”, “string”), structField(“b”, integer),
…)
read.df ( …, schema = schema)

From: Franc Carter [mailto:franc.carter@rozettatech.com]
Sent: Wednesday, August 19, 2015 1:48 PM
To: user@spark.apache.org
Subject: SparkR csv without headers


Hi,

Does anyone have an example of how to create a DataFrame in SparkR  which specifies the column
names - the csv files I have do not have column names in the first row. I can get read a csv
nicely with com.databricks:spark-csv_2.10:1.0.3, but I end up with column names C1, C2, C3
etc


thanks

--

Franc Carter     I      Systems Architect    I     RoZetta Technology



[Description: Description: Description: cid:image003.jpg@01D02903.9B540580]



L4. 55 Harrington Street,  THE ROCKS,  NSW, 2000

PO Box H58, Australia Square, Sydney NSW, 1215, AUSTRALIA

T  +61 2 8355 2515<tel:%2B61%202%208355%202515>     I    www.rozettatechnology.com<http://www.rozettatechnology.com/>

[cid:image002.jpg@01D02903.0B41B280]

DISCLAIMER: The contents of this email, inclusive of attachments, may be legally

privileged and confidential. Any unauthorised use of the contents is expressly prohibited.


Mime
View raw message