spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrian Bonar <>
Subject RE: Pre-build Spark for Windows 8.1
Date Thu, 05 Dec 2013 15:48:42 GMT
Excellent! Thank you, Matei.

From: Matei Zaharia []
Sent: Wednesday, December 4, 2013 4:26 PM
Subject: Re: Pre-build Spark for Windows 8.1

Hey Adrian,

Ideally you shouldn't use Cygwin to run on Windows - use the .cmd scripts we provide instead.
Cygwin might be made to work but we haven't tried to do this so far so it's not supported.
If you can fix it, that would of course be welcome.

Also, the deploy scripts don't work on Windows - we assumed that people would mostly use Windows
for local development and testing. However, if you do want to launch a cluster, you can use
spark-class.cmd manually to launch a Master and then Workers. Use the commands in the first
part of


On Dec 4, 2013, at 2:31 PM, Adrian Bonar <<>>

Separate from my previous thread about building a distribution of Spark on Win8, I am also
trying to run the pre-build binaries with little success. I downloaded and extract "spark-0.8.0-incubating-bin-hadoop1"
to d:\spark and attempted to start a master with the following error:

$ sh bin/
starting org.apache.spark.deploy.master.Master, logging to /cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out
failed to launch org.apache.spark.deploy.master.Master:

  Error: Could not find or load main class org.apache.spark.deploy.master.Master
full log in /cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out

--Adrian (again. :))

View raw message