spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: NoClassDefFoundError with Spark 1.3
Date Fri, 08 May 2015 08:40:24 GMT
Looks like the jar you provided has some missing classes. Try this:

scalaVersion := "2.10.4"

libraryDependencies ++= Seq(
    "org.apache.spark" %% "spark-core" % "1.3.0",
    "org.apache.spark" %% "spark-sql" % "1.3.0" % "provided",
    "org.apache.spark" %% "spark-mllib" % "1.3.0" % "provided",
    "log4j" % "log4j" % "1.2.15" excludeAll(
      ExclusionRule(organization = "com.sun.jdmk"),
      ExclusionRule(organization = "com.sun.jmx"),
      ExclusionRule(organization = "javax.jms")
      )
)


Thanks
Best Regards

On Thu, May 7, 2015 at 11:28 PM, Ganelin, Ilya <Ilya.Ganelin@capitalone.com>
wrote:

> Hi all – I’m attempting to build a project with SBT and run it on Spark
> 1.3 (this previously worked before we upgraded to CDH 5.4 with Spark 1.3).
>
> I have the following in my build.sbt:
>
>
> scalaVersion := "2.10.4"
>
> libraryDependencies ++= Seq(
>     "org.apache.spark" %% "spark-core" % "1.3.0" % "provided",
>     "org.apache.spark" %% "spark-sql" % "1.3.0" % "provided",
>     "org.apache.spark" %% "spark-mllib" % "1.3.0" % "provided",
>     "log4j" % "log4j" % "1.2.15" excludeAll(
>       ExclusionRule(organization = "com.sun.jdmk"),
>       ExclusionRule(organization = "com.sun.jmx"),
>       ExclusionRule(organization = "javax.jms")
>       )
> )
>
> When I attempt to run this program with sbt run, however, I get the
> following error:
> java.lang.NoClassDefFoundError: org.apache.spark.Partitioner
>
> I don’t explicitly use the Partitioner class anywhere, and this seems to
> indicate some missing Spark libraries on the install. Do I need to confirm
> anything other than the presence of the Spark assembly? I’m on CDH 5.4 and
> I’m able to run the spark-shell without any trouble.
>
> Any help would be much appreciated.
>
> Thank you,
> Ilya Ganelin
>
>
> ------------------------------
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed.  If the reader of this message is not the
> intended recipient, you are hereby notified that any review,
> retransmission, dissemination, distribution, copying or other use of, or
> taking of any action in reliance upon this information is strictly
> prohibited. If you have received this communication in error, please
> contact the sender and delete the material from your computer.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message