spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Tse <>
Subject Unit Testing Spark Transformations/Actions
Date Tue, 16 Jun 2015 16:19:40 GMT
Hi there,

I am looking to use Mockito to mock out some functionality while unit testing a Spark application.

I currently have code that happily runs on a cluster, but fails when I try to run unit tests
against it, throwing a "SparkException":

org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed
1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1, localhost): java.lang.ClassCastException:
cannot assign instance of java.lang.invoke.SerializedLambda to field$$anonfun$foreach$1.f$14
of type in instance of$$anonfun$foreach$1

(Full error/stacktrace and description on SO:

Has anyone experienced this error before while unit testing?


View raw message