spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From fabrizio silvestri <>
Subject SimpleJob.scala on a small cluster
Date Wed, 14 Aug 2013 21:47:46 GMT
Dear all,

I'm very new to Spark and I was trying to get acquainted with its basics
using the tests in the documents.

Let me state first that I'm able to run everything using spark-shell also
when I run it using a mesos master on a small cluster of 4 machines.

I've tried to compile and run the example contained in the "A Standalone
Job in Scala" section available here: "".

As long as it concerns to run the job locally, everything works.

I then modified the following line:

val sc = new SparkContext("local", "Simple Job", "$YOUR_SPARK_HOME",


val sc = new SparkContext("mesos://[4]", "Simple Job",
"$YOUR_SPARK_HOME", List("target/scala-2.9.3/simple-project_2.9.3-1.0.jar"))

to run on the above mentioned cluster of four machines.

Here's the issue:

sbt run

13/08/14 23:45:09 INFO spark.SparkContext: Added JAR
target/scala-2.9.3/simple-project_2.9.3-1.0.jar at
http://aaa.bbb.ccc.ddd:47683/jars/simple-project_2.9.3-1.0.jar with
timestamp 1376516709537
Failed to load native Mesos library from
[error] (run-main) java.lang.UnsatisfiedLinkError: no mesos in
java.lang.UnsatisfiedLinkError: no mesos in java.library.path
 at java.lang.ClassLoader.loadLibrary(
at java.lang.Runtime.loadLibrary0(
 at java.lang.System.loadLibrary(
at org.apache.mesos.MesosNativeLibrary.load(
 at spark.SparkContext.<init>(SparkContext.scala:170)
at SimpleJob$.main(SimpleJob.scala:8)
at SimpleJob.main(SimpleJob.scala)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(
13/08/14 23:45:09 INFO network.ConnectionManager: Selector thread was
java.lang.RuntimeException: Nonzero exit code: 1
at scala.sys.package$.error(package.scala:27)
[error] {file:/home/spark/DEMOs/SimpleJob/}default-031d15/compile:run:
Nonzero exit code: 1
[error] Total time: 4 s, completed Aug 14, 2013 11:45:09 PM

I've looked around and apparently I cannot find any help.

Thanks in advance

View raw message