spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gerard Maas <>
Subject Spark Job Server first steps
Date Thu, 22 May 2014 17:25:35 GMT

I'm starting to explore the Spark Job Server contributed by Ooyala [1],
running from the master branch.

I started by developing and submitting a simple job and the JAR check gave
me errors on a seemingly good jar. I disabled the fingerprint checking on
the jar and  I could submit it, but when I tried to submit the job, it
could not find it's classpath. Therefore I decided to take a couple of
steps backwards and go through the example in the docs.

Using the (Hello)WordCount example, upload is OK and the jar in in the UI
as well, but when I submit the job, I get the same classpathNotFound error
as before:

19:07 $ curl -d ""
  "status": "ERROR",
  "result": "classPath spark.jobserver.WordCountExample not found"

I'm not sure where it goes wrong.  Here's what seems to be the relevant
snippet in the server logs:

[2014-05-22 19:17:28,891] INFO  .apache.spark.SparkContext []
supervisor/666d021a-spark.jobserver.WordCountExample] - Added JAR
/tmp/spark-jobserver/filedao/data/test-2014-05-22T18:44:09.254+02:00.jar at with
timestamp 1400779048891
[2014-05-22 19:17:28,891] INFO  util.ContextURLClassLoader []
- Added URL
to ContextURLClassLoader
[2014-05-22 19:17:28,891] INFO  spark.jobserver.JarUtils$ []
- Loading object spark.jobserver.WordCountExample$ using loader
[2014-05-22 19:17:28,892] INFO  spark.jobserver.JarUtils$ []
- Loading class spark.jobserver.WordCountExample using loader

***** all OK until here and then ...*****

[2014-05-22 19:17:28,892] INFO  ocalContextSupervisorActor []
[akka://JobServer/user/context-supervisor] - Shutting down context

Any ideas? Something silly I might be doing?  btw, I'm running in dev mode
using sbt and default config (local).

-kr, Gerard.


View raw message