spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Krishnanand Khambadkone <kkhambadk...@yahoo.com.INVALID>
Subject Re: Spark Error: Cause was: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost:7077
Date Mon, 02 Mar 2015 23:18:54 GMT
I ran it with the --verbose option and I see this output

Using properties file: null
Parsed arguments:
  master                  spark://localhost:7077
  deployMode              cluster
  executorMemory          1g
  executorCores           null
  totalExecutorCores      null
  propertiesFile          null
  driverMemory            1g
  driverCores             null
  driverExtraClassPath    null
  driverExtraLibraryPath  null
  driverExtraJavaOptions  null
  supervise               false
  queue                   null
  numExecutors            100
  files                   null
  pyFiles                 null
  archives                null
  mainClass               KafkaMain
  primaryResource         file:/Users/hadoop/dev/kafkahbase/kafka/src/sparkhbase.jar
  name                    KafkaMain
  childArgs               []
  jars                    file:/Users/hadoop/kafka-0.8.1.1-src/core/build/libs/kafka_2.8.0-0.8.1.1.jar
  verbose                 true

Spark properties used, including those specified through
 --conf and those from the properties file null:
  

    
Main class:
org.apache.spark.deploy.Client
Arguments:
--memory
1g
launch
spark://localhost:7077
file:/Users/hadoop/dev/kafkahbase/kafka/src/sparkhbase.jar
KafkaMain
System properties:
spark.executor.memory -> 1g
SPARK_SUBMIT -> true
spark.app.name -> KafkaMain
spark.jars -> file:/Users/hadoop/kafka-0.8.1.1-src/core/build/libs/kafka_2.8.0-0.8.1.1.jar,file:/Users/hadoop/dev/kafkahbase/kafka/src/sparkhbase.jar
spark.master -> spark://localhost:7077
Classpath elements:

 

     On Monday, March 2, 2015 2:42 PM, Ted Yu <yuzhihong@gmail.com> wrote:
   

 bq. Cause was: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost:7077
There should be some more output following the above line.
Can you post them ?
Cheers
On Mon, Mar 2, 2015 at 2:06 PM, Krishnanand Khambadkone <kkhambadkone@yahoo.com.invalid>
wrote:

Hi,  I am running spark on my mac.   It is reading from a kafka topic and then writes the
data to a hbase table.  When I do a spark submit,  I get this error,
Error connecting to master spark://localhost:7077 (akka.tcp://sparkMaster@localhost:7077),
exiting.
Cause was: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost:7077
My submit statement looks like this,
./spark-submit --jars /Users/hadoop/kafka-0.8.1.1-src/core/build/libs/kafka_2.8.0-0.8.1.1.jar
--class KafkaMain --master spark://localhost:7077 --deploy-mode cluster --num-executors 100
--driver-memory 1g --executor-memory 1g /Users/hadoop/dev/kafkahbase/kafka/src/sparkhbase.jar





   
Mime
View raw message