spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yong Zhang <java8...@hotmail.com>
Subject Re: apply UDFs to N columns dynamically in dataframe
Date Thu, 16 Mar 2017 02:09:38 GMT
Is the answer here good for your case?


http://stackoverflow.com/questions/33151866/spark-udf-with-varargs

[https://cdn.sstatic.net/Sites/stackoverflow/img/apple-touch-icon@2.png?v=73d79a89bded]<http://stackoverflow.com/questions/33151866/spark-udf-with-varargs>

scala - Spark UDF with varargs - Stack Overflow<http://stackoverflow.com/questions/33151866/spark-udf-with-varargs>
stackoverflow.com
UDFs don't support varargs* but you can pass an arbitrary number of columns wrapped using
an array function: import org.apache.spark.sql.functions.{udf, array, lit ...





________________________________
From: anup ahire <ahireanup@gmail.com>
Sent: Wednesday, March 15, 2017 2:04 AM
To: user@spark.apache.org
Subject: apply UDFs to N columns dynamically in dataframe

Hello,

I have a schema and name of columns to apply UDF to. Name of columns are user input and they
can differ in numbers for each input.

Is there a way to apply UDFs to N columns in dataframe  ?



Thanks !

Mime
View raw message