spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Zhang <zjf...@gmail.com>
Subject Re: Submitted applications does not run.
Date Tue, 01 Sep 2015 08:11:07 GMT
Did you start spark cluster using command sbin/start-all.sh ?
You should have 2 log files under folder if it is single-node cluster. Like
the following

spark-jzhang-org.apache.spark.deploy.master.Master-1-jzhangMBPr.local.out
spark-jzhang-org.apache.spark.deploy.worker.Worker-1-jzhangMBPr.local.out



On Tue, Sep 1, 2015 at 4:01 PM, Madawa Soysa <madawa.11@cse.mrt.ac.lk>
wrote:

> There are no logs which includes apache.spark.deploy.worker in file name
> in the SPARK_HOME/logs folder.
>
> On 1 September 2015 at 13:00, Jeff Zhang <zjffdu@gmail.com> wrote:
>
>> This is master log. There's no worker registration info in the log. That
>> means the worker may not start properly. Please check the log file
>> with apache.spark.deploy.worker in file name.
>>
>>
>>
>> On Tue, Sep 1, 2015 at 2:55 PM, Madawa Soysa <madawa.11@cse.mrt.ac.lk>
>> wrote:
>>
>>> I cannot see anything abnormal in logs. What would be the reason for not
>>> availability of executors?
>>>
>>> On 1 September 2015 at 12:24, Madawa Soysa <madawa.11@cse.mrt.ac.lk>
>>> wrote:
>>>
>>>> Following are the logs available. Please find the attached.
>>>>
>>>> On 1 September 2015 at 12:18, Jeff Zhang <zjffdu@gmail.com> wrote:
>>>>
>>>>> It's in SPARK_HOME/logs
>>>>>
>>>>> Or you can check the spark web ui. http://[master-machine]:8080
>>>>>
>>>>>
>>>>> On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa <madawa.11@cse.mrt.ac.lk>
>>>>> wrote:
>>>>>
>>>>>> How do I check worker logs? SPARK_HOME/work folder does not exist.
I
>>>>>> am using the spark standalone mode.
>>>>>>
>>>>>> On 1 September 2015 at 12:05, Jeff Zhang <zjffdu@gmail.com>
wrote:
>>>>>>
>>>>>>> No executors ? Please check the worker logs if you are using
spark
>>>>>>> standalone mode.
>>>>>>>
>>>>>>> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa <
>>>>>>> madawa.11@cse.mrt.ac.lk> wrote:
>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I have successfully submitted some jobs to spark master.
But the
>>>>>>>> jobs won't progress and not finishing. Please see the attached
screenshot.
>>>>>>>> These are fairly very small jobs and this shouldn't take
more than a minute
>>>>>>>> to finish.
>>>>>>>>
>>>>>>>> I'm new to spark and any help would be appreciated.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Madawa.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>>>>>>> For additional commands, e-mail: user-help@spark.apache.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards
>>>>>>>
>>>>>>> Jeff Zhang
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *_________________________________**Madawa Soysa*
>>>>>>
>>>>>> Undergraduate,
>>>>>>
>>>>>> Department of Computer Science and Engineering,
>>>>>>
>>>>>> University of Moratuwa.
>>>>>>
>>>>>>
>>>>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>>>>> madawa.11@cse.mrt.ac.lk
>>>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter
>>>>>> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards
>>>>>
>>>>> Jeff Zhang
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *_________________________________**Madawa Soysa*
>>>>
>>>> Undergraduate,
>>>>
>>>> Department of Computer Science and Engineering,
>>>>
>>>> University of Moratuwa.
>>>>
>>>>
>>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>>> madawa.11@cse.mrt.ac.lk
>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter
>>>> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/>
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> *_________________________________**Madawa Soysa*
>>>
>>> Undergraduate,
>>>
>>> Department of Computer Science and Engineering,
>>>
>>> University of Moratuwa.
>>>
>>>
>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>> madawa.11@cse.mrt.ac.lk
>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter
>>> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/>
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
>
> *_________________________________**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa.11@cse.mrt.ac.lk
> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter
> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/>
>



-- 
Best Regards

Jeff Zhang

Mime
View raw message