spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Borckmans <pierre.borckm...@realimpactanalytics.com>
Subject Re: Spark 0.9.0 - local mode - sc.addJar problem (bug?)
Date Sun, 02 Mar 2014 14:59:51 GMT
Hi Nan,

Must be a local network config problem with my machine.
When I replace the 10.0.1.7 ip with localhost, it works perfectly…
Thanks for trying though…

Pierre

On 02 Mar 2014, at 16:02, Nan Zhu <zhunanmcgill@gmail.com> wrote:

> Cannot reproduce it….even I add spark-assembly jar
> 
> scala> sc.addJar("/Users/nanzhu/code/spark-0.9.0-incubating/assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-hadoop1.0.4.jar")
> 14/03/02 09:59:47 INFO SparkContext: Added JAR /Users/nanzhu/code/spark-0.9.0-incubating/assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-hadoop1.0.4.jar
at http://192.168.2.17:55364/jars/spark-assembly-0.9.0-incubating-hadoop1.0.4.jar with timestamp
1393772387586
> 
> 
> --2014-03-02 09:59:54--  http://192.168.2.17:55364/jars/spark-assembly-0.9.0-incubating-hadoop1.0.4.jar
> Connecting to 192.168.2.17:55364... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 87878749 (84M) [application/java-archive]
> Saving to: ‘spark-assembly-0.9.0-incubating-hadoop1.0.4.jar.1’
> 
> 100%[========================================================================>] 87,878,749
 66.7MB/s   in 1.3s   
> 
> 2014-03-02 09:59:56 (66.7 MB/s) - ‘spark-assembly-0.9.0-incubating-hadoop1.0.4.jar.1’
saved [87878749/87878749]
> 
> 
> Best,
> 
> -- 
> Nan Zhu
> 
> On Sunday, March 2, 2014 at 9:48 AM, Pierre B wrote:
> 
>> Hi all!
>> 
>> In spark 0.9.0, local mode, whenever I try to add jar(s), using either
>> SparkConf.addJars or SparkConfiguration.addJar, in the shell or in a
>> standalone mode, I observe a strange behaviour.
>> 
>> I investigated this because my standalone app works perfectly on my cluster
>> but is getting stuck in local mode.
>> 
>> So, as can be seen in the following screenshot, the jar file is supposedly
>> made available at the given http address:
>> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n2218/Screen_Shot_2014-03-02_at_15.30.51.png>
>> 
>> 
>> However, when I try to get the file from http (in a browser or using wget),
>> the download always gets stuck after a while (usually around 66,724 bytes,
>> as can be seen in the next screenshot):
>> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n2218/Screen_Shot_2014-03-02_at_15.30.39.png>
>> 
>> This happens for all jars I've trying, except the ones smaller than
>> 60kbytes.
>> 
>> Could this be a bug or just a problem on my machine? (MacOS Mavericks)
>> 
>> Cheers
>> 
>> Pierre
>> 
>> 
>> 
>> --
>> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-0-9-0-local-mode-sc-addJar-problem-bug-tp2218.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
> 


Mime
View raw message