spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhu <ma...@madhu.com>
Subject Re: java.lang.NoSuchMethodError on Java API
Date Tue, 13 May 2014 00:21:34 GMT
I was able to compile your code in Eclipse.
I ran it using the data in your comments, but I also see the
NoSuchMethodError you mentioned.

It seems to run fine until the call to calculateZVector(...)
It appears that org.apache.commons.math3.util.Pair is not Serializable, so
that's one potential problem.
I created a Serializable version of Pair, but that wasn't enough.

Commenting this code:

zVectorRaw.reduce(new Function2<Tuple2&lt;Integer, Double>, Tuple2<Integer,
Double>, Tuple2<Integer, Double>>() {
	@Override
	public Tuple2<Integer, Double> call(Tuple2<Integer, Double>
integerDoubleTuple2, Tuple2<Integer, Double> integerDoubleTuple22) throws
Exception {
		return null;
	}
});

Avoids the NoSuchMethodError, so that might be part of your problem.

Then I get a NotSerializableException, so my guess is there's a reference to
something else that's not serializable in that's referenced in that method.
There's a lot of stuff going on in that method, so it's not easy for me to
follow.

I would break it down to more manageable pieces and build it up one step at
a time.

Sorry I couldn't find the problem.



-----
Madhu
https://www.linkedin.com/in/msiddalingaiah
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-lang-NoSuchMethodError-on-Java-API-tp5545p5623.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message