spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <AShaita...@nz.imshealth.com>
Subject Adding a new nested column into dataframe
Date Wed, 07 Dec 2016 05:09:26 GMT
Hello Spark experts!

I need to add one more nested column to existing ones. For example:

Initial DF schema looks like that:

|-- Id: string (nullable = true)
|-- Type: string (nullable = true)
|-- Uri: string (nullable = true)
|-- attributes: struct (nullable = false)
| |-- CountryGroupID: array (nullable = true)
| | |-- element: string (containsNull = true)
| |-- CountryGroupDesc: array (nullable = true)
| | |-- element: string (containsNull = true)

But I want to add one more nested column:

|-- Id: string (nullable = true)
|-- Type: string (nullable = true)
|-- Uri: string (nullable = true)
|-- attributes: struct (nullable = false)
| |-- CountryGroupID: array (nullable = true)
| | |-- element: string (containsNull = true)
| |-- CountryGroupDesc: array (nullable = true)
| | |-- element: string (containsNull = true)
| |-- Status: array (nullable = true)
| | |-- element: string (containsNull = true)

I am working on Spark 1.6.2. I would be really grateful for any help.

Thank you in advance.


Best regards,

Artem


________________________________
********************** IMPORTANT--PLEASE READ ************************ This electronic message,
including its attachments, is CONFIDENTIAL and may contain PROPRIETARY or LEGALLY PRIVILEGED
or PROTECTED information and is intended for the authorized recipient of the sender. If you
are not the intended recipient, you are hereby notified that any use, disclosure, copying,
or distribution of this message or any of the information included in it is unauthorized and
strictly prohibited. If you have received this message in error, please immediately notify
the sender by reply e-mail and permanently delete this message and its attachments, along
with any copies thereof, from all locations received (e.g., computer, mobile device, etc.).
Thank you. ************************************************************************

Mime
View raw message