spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Diana Carroll <>
Subject quick start guide: building a standalone scala program
Date Mon, 24 Mar 2014 19:44:41 GMT
Has anyone successfully followed the instructions on the Quick Start page
of the Spark home page to run a "standalone" Scala application?  I can't,
and I figure I must be missing something obvious!

I'm trying to follow the instructions here as close to "word for word" as

1.  The instructions don't say what directory to create my test application
in, but later I'm instructed to run "sbt/sbt" so I conclude that my working
directory must be $SPARK_HOME.  (Temporarily ignoring that it is a little
weird to be working directly in the Spark distro.)

2.  Create $SPARK_HOME/mysparktest/src/main/scala/SimpleApp.scala.
 Copy&paste in the code from the instructions exactly, replacing
YOUR_SPARK_HOME with my spark home path.

3.  Create $SPARK_HOME/mysparktest/simple.sbt.  Copy&paste in the sbt file
from the instructions

4.  From the $SPARK_HOME I run "sbt/sbt package".  It runs through the
ENTIRE Spark project!  This takes several minutes, and at the end, it says
"Done packaging".  unfortunately, there's nothing in the
$SPARK_HOME/mysparktest/ folder other than what I already had there.

(Just for fun, I also did what I thought was more logical, which is set my
working directory to $SPARK_HOME/mysparktest, and but $SPARK_HOME/sbt/sbt
package, but that was even less successful: I got an error:
awk: cmd. line:1: fatal: cannot open file `./project/' for
reading (No such file or directory)
Attempting to fetch sbt
/usr/lib/spark/sbt/sbt: line 33: sbt/sbt-launch-.jar: No such file or
/usr/lib/spark/sbt/sbt: line 33: sbt/sbt-launch-.jar: No such file or
Our attempt to download sbt locally to sbt/sbt-launch-.jar failed. Please
install sbt manually from

So, help?  I'm sure these instructions work because people are following
them every day, but I can't tell what they are supposed to do.


View raw message