spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Proust GZ Feng <pf...@cn.ibm.com>
Subject Re: NO Cygwin Support in bin/spark-class in Spark 1.4.0
Date Tue, 28 Jul 2015 08:49:57 GMT
Thanks Owen, the problem under Cygwin is while run spark-submit under 
1.4.0, it simply report

Error: Could not find or load main class org.apache.spark.launcher.Main

This is because under Cygwin spark-class make the LAUNCH_CLASSPATH as "
/c/spark-1.4.0-bin-hadoop2.3/lib/spark-assembly-1.4.0-hadoop2.3.0.jar"
But under Cygwin java in Windows cannot recognize the classpath, so below 
command simply error out

 java -cp 
/c/spark-1.4.0-bin-hadoop2.3/lib/spark-assembly-1.4.0-hadoop2.3.0.jar 
org.apache.spark.launcher.Main
Error: Could not find or load main class org.apache.spark.launcher.Main

Thanks
Proust



From:   Sean Owen <sowen@cloudera.com>
To:     Proust GZ Feng/China/IBM@IBMCN
Cc:     user <user@spark.apache.org>
Date:   07/28/2015 02:20 PM
Subject:        Re: NO Cygwin Support in bin/spark-class in Spark 1.4.0



It wasn't removed, but rewritten. Cygwin is just a distribution of
POSIX-related utilities so you should be able to use the normal .sh
scripts. In any event, you didn't say what the problem is?

On Tue, Jul 28, 2015 at 5:19 AM, Proust GZ Feng <pfeng@cn.ibm.com> wrote:
> Hi, Spark Users
>
> Looks like Spark 1.4.0 cannot work with Cygwin due to the removing of 
Cygwin
> support in bin/spark-class
>
> The changeset is
> 
https://github.com/apache/spark/commit/517975d89d40a77c7186f488547eed11f79c1e97#diff-fdf4d3e600042c63ffa17b692c4372a3

>
> The changeset said "Add a library for launching Spark jobs
> programmatically", but how to use it in Cygwin?
> I'm wondering any solutions available to make it work in Windows?
>
>
> Thanks
> Proust

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org



Mime
View raw message