spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From alexeys <alex...@princeton.edu>
Subject VectorUDT with spark.ml.linalg.Vector
Date Tue, 16 Aug 2016 16:48:07 GMT
I am writing an UDAF to be applied to a data frame column of type Vector
(spark.ml.linalg.Vector). I rely on spark/ml/linalg so that I do not have to
go back and forth between dataframe and RDD. 

Inside the UDAF, I have to specify a data type for the input, buffer, and
output (as usual). VectorUDT is what I would use with
spark.mllib.linalg.Vector: 
https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala

However, when I try to import it from spark.ml instead: import
org.apache.spark.ml.linalg.VectorUDT 
I get a runtime error (no errors during the build): 

class VectorUDT in package linalg cannot be accessed in package
org.apache.spark.ml.linalg 

Is it expected/can you suggest a workaround? 

I am using Spark 2.0.0

Thanks, 
Alexey



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/VectorUDT-with-spark-ml-linalg-Vector-tp27542.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message