spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Zhang <zjf...@gmail.com>
Subject Re: Submitted applications does not run.
Date Tue, 01 Sep 2015 08:42:40 GMT
You need to make yourself able to ssh to localhost without password, please
check this blog.

http://hortonworks.com/kb/generating-ssh-keys-for-passwordless-login/



On Tue, Sep 1, 2015 at 4:31 PM, Madawa Soysa <madawa.11@cse.mrt.ac.lk>
wrote:

> I used ./sbin/start-master.sh
>
> When I used ./sbin/start-all.sh the start fails. I get the following error.
>
> failed to launch org.apache.spark.deploy.master.Master:
> localhost: ssh: connect to host localhost port 22: Connection refused
>
> On 1 September 2015 at 13:41, Jeff Zhang <zjffdu@gmail.com> wrote:
>
>> Did you start spark cluster using command sbin/start-all.sh ?
>> You should have 2 log files under folder if it is single-node cluster.
>> Like the following
>>
>> spark-jzhang-org.apache.spark.deploy.master.Master-1-jzhangMBPr.local.out
>> spark-jzhang-org.apache.spark.deploy.worker.Worker-1-jzhangMBPr.local.out
>>
>>
>>
>> On Tue, Sep 1, 2015 at 4:01 PM, Madawa Soysa <madawa.11@cse.mrt.ac.lk>
>> wrote:
>>
>>> There are no logs which includes apache.spark.deploy.worker in file
>>> name in the SPARK_HOME/logs folder.
>>>
>>> On 1 September 2015 at 13:00, Jeff Zhang <zjffdu@gmail.com> wrote:
>>>
>>>> This is master log. There's no worker registration info in the log.
>>>> That means the worker may not start properly. Please check the log file
>>>> with apache.spark.deploy.worker in file name.
>>>>
>>>>
>>>>
>>>> On Tue, Sep 1, 2015 at 2:55 PM, Madawa Soysa <madawa.11@cse.mrt.ac.lk>
>>>> wrote:
>>>>
>>>>> I cannot see anything abnormal in logs. What would be the reason for
>>>>> not availability of executors?
>>>>>
>>>>> On 1 September 2015 at 12:24, Madawa Soysa <madawa.11@cse.mrt.ac.lk>
>>>>> wrote:
>>>>>
>>>>>> Following are the logs available. Please find the attached.
>>>>>>
>>>>>> On 1 September 2015 at 12:18, Jeff Zhang <zjffdu@gmail.com>
wrote:
>>>>>>
>>>>>>> It's in SPARK_HOME/logs
>>>>>>>
>>>>>>> Or you can check the spark web ui. http://[master-machine]:8080
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa <
>>>>>>> madawa.11@cse.mrt.ac.lk> wrote:
>>>>>>>
>>>>>>>> How do I check worker logs? SPARK_HOME/work folder does not
exist.
>>>>>>>> I am using the spark standalone mode.
>>>>>>>>
>>>>>>>> On 1 September 2015 at 12:05, Jeff Zhang <zjffdu@gmail.com>
wrote:
>>>>>>>>
>>>>>>>>> No executors ? Please check the worker logs if you are
using spark
>>>>>>>>> standalone mode.
>>>>>>>>>
>>>>>>>>> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa <
>>>>>>>>> madawa.11@cse.mrt.ac.lk> wrote:
>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> I have successfully submitted some jobs to spark
master. But the
>>>>>>>>>> jobs won't progress and not finishing. Please see
the attached screenshot.
>>>>>>>>>> These are fairly very small jobs and this shouldn't
take more than a minute
>>>>>>>>>> to finish.
>>>>>>>>>>
>>>>>>>>>> I'm new to spark and any help would be appreciated.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Madawa.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>>>>>>>>> For additional commands, e-mail: user-help@spark.apache.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards
>>>>>>>>>
>>>>>>>>> Jeff Zhang
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> *_________________________________**Madawa Soysa*
>>>>>>>>
>>>>>>>> Undergraduate,
>>>>>>>>
>>>>>>>> Department of Computer Science and Engineering,
>>>>>>>>
>>>>>>>> University of Moratuwa.
>>>>>>>>
>>>>>>>>
>>>>>>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> |
Email:
>>>>>>>> madawa.11@cse.mrt.ac.lk
>>>>>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> |
Twitter
>>>>>>>> <https://twitter.com/madawa_rc> | Tumblr
>>>>>>>> <http://madawas.tumblr.com/>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards
>>>>>>>
>>>>>>> Jeff Zhang
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *_________________________________**Madawa Soysa*
>>>>>>
>>>>>> Undergraduate,
>>>>>>
>>>>>> Department of Computer Science and Engineering,
>>>>>>
>>>>>> University of Moratuwa.
>>>>>>
>>>>>>
>>>>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>>>>> madawa.11@cse.mrt.ac.lk
>>>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter
>>>>>> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *_________________________________**Madawa Soysa*
>>>>>
>>>>> Undergraduate,
>>>>>
>>>>> Department of Computer Science and Engineering,
>>>>>
>>>>> University of Moratuwa.
>>>>>
>>>>>
>>>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>>>> madawa.11@cse.mrt.ac.lk
>>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter
>>>>> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> *_________________________________**Madawa Soysa*
>>>
>>> Undergraduate,
>>>
>>> Department of Computer Science and Engineering,
>>>
>>> University of Moratuwa.
>>>
>>>
>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>> madawa.11@cse.mrt.ac.lk
>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter
>>> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/>
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
>
> *_________________________________**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa.11@cse.mrt.ac.lk
> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter
> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/>
>



-- 
Best Regards

Jeff Zhang

Mime
View raw message