spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ramkumar Venkataraman <>
Subject [Spark Streaming] NoClassDefFoundError : StateSpec
Date Thu, 12 Jan 2017 15:35:10 GMT
Spark: 1.6.1

I am trying to use the new mapWithState API and I am getting the following

Exception in thread "main" java.lang.NoClassDefFoundError:
Caused by: java.lang.ClassNotFoundException:

scalaVersion := "2.10.6"
val sparkVersion = "1.6.1"

resolvers ++= Seq(
  "Sonatype OSS Snapshots" at

libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-streaming-kafka" % sparkVersion,
  "com.fasterxml.jackson.core" % "jackson-databind" % "2.3.3" // Needed by

This is how my spark-submit looks like:

./bin/spark-submit --verbose --master yarn-client  --num-executors 50
--driver-memory=4G --executor-memory=8G   --conf
"spark.driver.extraJavaOptions=-XX:MaxPermSize=6G -XX:+UseConcMarkSweepGC"
--conf "spark.executor.extraJavaOptions=-XX:+UseConcMarkSweepGC -verbose:gc
-XX:+PrintGCDetails -XX:+PrintGCTimeStamps"  --class MY_DRIVER


Is there anything I am missing here? I understand that NoClassDefFoundError
means the required Jars aren't present in the classpath, I am just not able
to understand why this class alone is missing, when the others related to
window, etc. are found. Do I have to pass in additional jars to make this
API work?

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe e-mail:

View raw message