spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ramkumar Chokkalingam <ramkumar...@gmail.com>
Subject Program hangs when using local[2]
Date Wed, 30 Oct 2013 08:20:46 GMT
Hello Community,

I'm running Spark on a single Server in a stand-alone mode . The Server has
*two cores*  and so while initializing the SparkContext object, I used

*var sc = new SparkContext("local[2]", "MyApp", SPARK_HOME)*

 [ As per this
http://spark.incubator.apache.org/docs/0.6.0/scala-programming-guide.html#master-urls
]

But, the program while processing files of size more than 60MB hangs. This
without emitting any error message or anything(I have my Heap size
increased to large size). Also, when I checked *the _temporary directory * in
output location, I noticed that the program has stopped doing any further
processing/parsing of my input directory.

Any ideas on why it happens when I use "local[2]" ?

Regards,

Ramkumar Chokkalingam ,
University of Washington.
LinkedIn <http://www.linkedin.com/in/mynameisram>

Mime
View raw message