spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matteo Cossu <elco...@gmail.com>
Subject Re: can udaf's return complex types?
Date Tue, 13 Feb 2018 09:38:19 GMT
Hello,

yes, sure they can return complex types. For example, the functions
collect_list and collect_set return an ArrayType.

On 10 February 2018 at 14:28, kant kodali <kanth909@gmail.com> wrote:

> Hi All,
>
> Can UDAF's return complex types? like say a Map with key as an Integer and
> the value as an Array of strings?
>
> For Example say I have the following *input dataframe*
>
> id | name | amount
> -------------------------
> 1 |  foo     | 10
> 2 |  bar     | 15
> 1 |  car     | 20
> 1 |  bus     | 20
>
> and my *target/output data frame* is
>
> id | my_sum_along_with_names
> -------------------------------------------------------------
> 1  | Map(key -> 50, value -> [foo, car, bus])
> 2  | Map(key -> 15, value -> [bar])
>
> I am looking for a UDAF solution so I can use it in my raw sql query.
>
> Thanks!
>
>
>
>
>
>
>
>
>

Mime
View raw message