spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheng Lian <lian.cs....@gmail.com>
Subject Re: How to insert complex types like map<string,map<string,int>> in spark sql
Date Thu, 27 Nov 2014 04:51:09 GMT
Thanks Takuya! Will take a look into it later. And sorry for not being 
able to review all the PRs in time recently (mostly because of rushing 
Spark 1.2 release and Thanksgiving :) ).

On 11/27/14 1:35 AM, Takuya UESHIN wrote:
> Hi,
> I guess this is fixed by https://github.com/apache/spark/pull/3110
> which is not for complex type casting but makes inserting into hive
> table be able to handle complex types ignoring nullability.
>
> I also sent a pull-request (https://github.com/apache/spark/pull/3150)
> for complex type casting before.
> Please check it if #3110 doesn't work.
>
> Thanks.
>
>
> 2014-11-25 18:37 GMT+09:00 Cheng Lian <lian.cs.zju@gmail.com>:
>> Spark SQL supports complex types, but casting doesn't work for complex types
>> right now.
>>
>>
>> On 11/25/14 4:04 PM, critikaled wrote:
>>>
>>> https://github.com/apache/spark/blob/84d79ee9ec47465269f7b0a7971176da93c96f3f/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
>>>
>>> Doesn't look like spark sql support nested complex types right now
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-insert-complex-types-like-map-string-map-string-int-in-spark-sql-tp19603p19730.html
>>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: user-help@spark.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message