spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ayan guha <guha.a...@gmail.com>
Subject Re: Windows - Spark 2 - Standalone - Worker not able to connect to Master
Date Mon, 01 Aug 2016 11:35:35 GMT
No I confirmed master is running by spark ui at localhost:8080
On 1 Aug 2016 18:22, "Nikolay Zhebet" <phpapple@gmail.com> wrote:

> I think you haven't run spark master yet, or maybe port 7077 is not yours
> default port for spark master.
>
> 2016-08-01 4:24 GMT+03:00 ayan guha <guha.ayan@gmail.com>:
>
>> Hi
>>
>> I just downloaded Spark 2.0 on my windows 7 to check it out. However, not
>> able to set up a standalone cluster:
>>
>>
>> Step 1: master set up (Successful)
>>
>> bin/spark-class org.apache.spark.deploy.master.Master
>>
>> It did throw an error about not able to find winutils, but started
>> successfully.
>>
>> Step II: Set up Worker (Failed)
>>
>> bin/spark-class org.apache.spark.deploy.worker.Worker
>> spark://localhost:7077
>>
>> This step fails with following error:
>>
>> 16/08/01 11:21:27 INFO Worker: Connecting to master localhost:7077...
>> 16/08/01 11:21:28 WARN Worker: Failed to connect to master localhost:7077
>> org.apache.spark.SparkException: Exception thrown in awaitResult
>>         at
>> org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.sca
>> la:77)
>>         at
>> org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.sca
>> la:75)
>>         at
>> scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.s
>> cala:36)
>>         at
>> org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyO
>> rElse(RpcTimeout.scala:59)
>>         at
>> org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyO
>> rElse(RpcTimeout.scala:59)
>>         at scala.PartialFunction$OrElse.apply(PartialFunction.scala:167)
>>         at
>> org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:83)
>>         at
>> org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:88)
>>         at org.apache.spark.rpc.RpcEnv.setupEndpointRef(RpcEnv.scala:96)
>>         at
>> org.apache.spark.deploy.worker.Worker$$anonfun$org$apache$spark$deplo
>> y$worker$Worker$$tryRegisterAllMasters$1$$anon$1.run(Worker.scala:216)
>>         at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
>> Source)
>>         at java.util.concurrent.FutureTask.run(Unknown Source)
>>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
>> Source)
>>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>> Source)
>>         at java.lang.Thread.run(Unknown Source)
>> Caused by: java.io.IOException: Failed to connect to localhost/
>> 127.0.0.1:7077
>>         at
>> org.apache.spark.network.client.TransportClientFactory.createClient(T
>> ransportClientFactory.java:228)
>>         at
>> org.apache.spark.network.client.TransportClientFactory.createClient(T
>> ransportClientFactory.java:179)
>>         at
>> org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala
>> :197)
>>         at
>> org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:191)
>>         at
>> org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:187)
>>         ... 4 more
>> Caused by: java.net.ConnectException: Connection refused: no further
>> information
>> : localhost/127.0.0.1:7077
>>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>         at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
>>         at
>> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocke
>> tChannel.java:224)
>>         at
>> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConne
>> ct(AbstractNioChannel.java:289)
>>         at
>> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.jav
>> a:528)
>>         at
>> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEve
>> ntLoop.java:468)
>>         at
>> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.ja
>> va:382)
>>         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
>>         at
>> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThread
>> EventExecutor.java:111)
>>         ... 1 more
>>
>> Am I doing something wrong?
>>
>>
>> --
>> Best Regards,
>> Ayan Guha
>>
>
>

Mime
View raw message