spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Meethu Mathew <>
Subject Python to Java object conversion of numpy array
Date Fri, 09 Jan 2015 12:45:28 GMT
I am trying to send a numpy array as an argument to a function predict() 
in a class in spark/python/pyspark/mllib/ which is passed 
to the function callMLlibFunc(name, *args)  in 

Now the value is passed to the function  _py2java(sc, obj) .Here I am 
getting an exception

Py4JJavaError: An error occurred while calling z:org.apache.spark.mllib.api.python.SerDe.loads.
: net.razorvine.pickle.PickleException: expected zero arguments for construction of ClassDict
(for numpy.core.multiarray._reconstruct)
	at net.razorvine.pickle.objects.ClassDictConstructor.construct(
	at net.razorvine.pickle.Unpickler.load_reduce(
	at net.razorvine.pickle.Unpickler.dispatch(
	at net.razorvine.pickle.Unpickler.load(
	at net.razorvine.pickle.Unpickler.loads(

Why common._py2java(sc, obj) is not handling numpy array type?

Please help..



*Meethu Mathew*


*Flytxt* | Visit our blog <> | Follow us 
<> | _Connect on Linkedin 

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message