spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ogoh <oke...@gmail.com>
Subject using UDF( defined in Java) in scala through scala
Date Tue, 29 Sep 2015 23:06:03 GMT
Hello,
I have a udf declared in Java but I'd like to call it from spark-shell which
only supports Scala.
Since I am new to Scala, I couldn't figure out how to call register the Java
UDF using sqlContext.udf.register in scala.
Below is how I tried.
I appreciate any help.
Thanks,
----
= my UDF in java
public class ArrayStringOfJson implements UDF1<String, ArrayType> {
    public ArrayType call(String input) throws Exception {
    ..
  }
}
= using in spark-shell
scala> import org.apache.spark.sql.api.java.UDF1
scala> import com.mycom.event.udfs.ArrayStringOfJson
scala> val sqlContext = new org.apache.spark.sql.SQLContext(sc)
scala> val instance: ArrayStringOfJson = new ArrayStringOfJson
scala> sqlContext.udf.register("arraystring", instance,
org.apache.spark.sql.types.ArrayType)
<console>:28: error: overloaded method value register with alternatives:
  (name: String,f: org.apache.spark.sql.api.java.UDF22[_, _, _, _, _, _, _,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _],returnType:
org.apache.spark.sql.types.DataType)Unit <and>
  ...
  (name: String,f: org.apache.spark.sql.api.java.UDF1[_, _],returnType:
org.apache.spark.sql.types.DataType)Unit
 cannot be applied to (String, com.mycom.event.udfs.ArrayStringOfJson,
org.apache.spark.sql.types.ArrayType.type)
              sqlContext.udf.register("arraystring", instance,
org.apache.spark.sql.types.ArrayType)
                             ^





--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/using-UDF-defined-in-Java-in-scala-through-scala-tp24880.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message