spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe L <>
Subject Re: java.lang.ClassNotFoundException
Date Fri, 02 May 2014 03:52:59 GMT
Hi, You should include the jar file of your project. for example: conf.set("yourjarfilepath.jar")

On Friday, May 2, 2014 7:39 AM, proofmoore [via Apache Spark User List] <>
HelIo. I followed "A Standalone App in Java" part of the tutorial

Spark standalone cluster looks it's running without a problem :

I have built a fat jar for running this JavaApp on the cluster. Before maven package: 
    find .

content of is :

     import org.apache.spark.SparkConf;
     import org.apache.spark.SparkContext;

     public class SimpleApp {
     public static void main(String[] args) {

     SparkConf conf =  new SparkConf()
                       .setAppName("My app")
                       .set("spark.executor.memory", "1g");

     JavaSparkContext   sc = new JavaSparkContext (conf);
     String logFile = "/home/ubuntu/spark-0.9.1/test_data";
     JavaRDD<String> logData = sc.textFile(logFile).cache();

     long numAs = logData.filter(new Function<String, Boolean>() {
      public Boolean call(String s) { return s.contains("a"); }

     System.out.println("Lines with a: " + numAs); 
This program only works when master is set as setMaster("local"). Otherwise I get this error


If you reply to this email, your message will be added to the discussion below:

To start a new topic under Apache Spark User List, email

To unsubscribe from Apache Spark User List, click here.

View this message in context:
Sent from the Apache Spark User List mailing list archive at
View raw message