spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: is there a way of register python UDF using java API?
Date Mon, 02 Apr 2018 19:11:52 GMT
Looks like there is spark.udf().registerPython() like below.

public void registerPython(java.lang.String name,
org.apache.spark.sql.execution.python.UserDefinedPythonFunction udf)


can anyone describe what *udfDeterministic *parameter does in the method
signature below?

public UserDefinedPythonFunction(java.lang.String name,
org.apache.spark.api.python.PythonFunction func,
org.apache.spark.sql.types.DataType dataType, int pythonEvalType,
boolean udfDeterministic) { /* compiled code */ }


On Sun, Apr 1, 2018 at 3:46 PM, kant kodali <kanth909@gmail.com> wrote:

> Hi All,
>
> All of our spark code is in Java wondering if there a way to register
> python UDF's using java API such that the registered UDF's can be used
> using raw spark SQL.
> If there is any other way to achieve this goal please suggest!
>
> Thanks
>
>

Mime
View raw message