spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Somasundaram Sekar <somasundar.se...@tigeranalytics.com>
Subject Re: Spark transformations
Date Sun, 04 Sep 2016 16:16:52 GMT
Can you try this

https://www.linkedin.com/pulse/hive-functions-udfudaf-udtf-examples-gaurav-singh

On 4 Sep 2016 9:38 pm, "janardhan shetty" <janardhanp22@gmail.com> wrote:

> Hi,
>
> Is there any chance that we can send entire multiple columns to an udf and
> generate a new column for Spark ML.
> I see similar approach as VectorAssembler but not able to use few classes
> /traitslike HasInputCols, HasOutputCol, DefaultParamsWritable since they
> are private.
>
> Any leads/examples is appreciated in this regard..
>
> Requirement:
> *Input*: Multiple columns of a Dataframe
> *Output*:  Single new modified column
>

Mime
View raw message