spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheng Lian <lian.cs....@gmail.com>
Subject Re: MapType in spark-sql
Date Wed, 21 Jan 2015 02:19:15 GMT
You need to provide key type, value type for map type, element type for 
array type, and whether they contain null:

|StructType(Array(
   StructField("map_field",MapType(keyType =IntegerType, valueType =StringType, containsNull
=true), nullable =true),
   StructField("array_field",ArrayType(elementType =DoubleType, containsNull =true), nullable
=true)))
|

Cheng

On 1/20/15 5:50 PM, Kevin Jung wrote:

> Hi all
> How can I add MapType and ArrayType to schema when I create StructType
> programmatically?
> val schema =
>    StructType(
>      schemaString.split(" ").map(fieldName => StructField(fieldName,
> StringType, true)))
> above code from spark document works fine but if I change StringType to
> MapType or ArrayType , it isn't compiled. Thanks in advance.
>
> Kevin
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/MapType-in-spark-sql-tp21274.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
‚Äč

Mime
View raw message