spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imran Rajjad <raj...@gmail.com>
Subject task not serializable on simple operations
Date Mon, 16 Oct 2017 08:21:06 GMT
Is there a way around to implement a separate Java class that implements
serializable interface for even small petty arithmetic operations?

below is code from simple decision tree example

Double testMSE =  predictionAndLabel.map(new Function<Tuple2<Double,
Double>, Double>() {
   @Override
      public Double call(Tuple2<Double, Double> pl) {
        Double diff = pl._1() - pl._2();
        return diff * diff;
      }
    }).reduce(new Function2<Double, Double, Double>() {

   @Override
      public Double call(Double a, Double b) {
        return a + b;
      }
    }) / testDataCount;

there are no complex java objects involved here and simple arithmetic
operations.

regards,
Imran
-- 
I.R

Mime
View raw message