spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jestin Ma <jestinwith.a...@gmail.com>
Subject Re: Spark Web UI port 4040 not working
Date Tue, 26 Jul 2016 13:50:04 GMT
Also, sorry for the repeated updates, I checked my master UI and it says no
drivers are running but 1 application is running.

On Tue, Jul 26, 2016 at 6:49 AM, Jestin Ma <jestinwith.an.e@gmail.com>
wrote:

> I did netstat -apn | grep 4040 on machine 6, and I see
>
> tcp        0      0 :::4040                     :::*
>  LISTEN      30597/java
>
> What does this mean?
>
> On Tue, Jul 26, 2016 at 6:47 AM, Jestin Ma <jestinwith.an.e@gmail.com>
> wrote:
>
>> I do not deploy using cluster mode and I don't use EC2.
>>
>> I just read that launching as client mode: "the driver is launched
>> directly within the spark-submit process which acts as a *client* to the
>> cluster."
>>
>> My current setup is that I have cluster machines 1, 2, 3, 4, 5, with 1
>> being the master.
>> I submit from another cluster machine 6 in client mode. So I'm taking
>> that the driver is launched in my machine 6.
>>
>> On Tue, Jul 26, 2016 at 6:38 AM, Jacek Laskowski <jacek@japila.pl> wrote:
>>
>>> Hi,
>>>
>>> Do you perhaps deploy using cluster mode? Is this EC2? You'd need to
>>> figure out where the driver runs and use the machine's IP.
>>>
>>> Pozdrawiam,
>>> Jacek Laskowski
>>> ----
>>> https://medium.com/@jaceklaskowski/
>>> Mastering Apache Spark http://bit.ly/mastering-apache-spark
>>> Follow me at https://twitter.com/jaceklaskowski
>>>
>>>
>>> On Tue, Jul 26, 2016 at 3:36 PM, Jestin Ma <jestinwith.an.e@gmail.com>
>>> wrote:
>>> > I tried doing that on my master node.
>>> > I got nothing.
>>> > However, I grep'd port 8080 and I got the standalone UI.
>>> >
>>> > On Tue, Jul 26, 2016 at 12:39 AM, Chanh Le <giaosudau@gmail.com>
>>> wrote:
>>> >>
>>> >> You’re running in StandAlone Mode?
>>> >> Usually inside active task it will show the address of current job.
>>> >> or you can check in master node by using netstat -apn | grep 4040
>>> >>
>>> >>
>>> >>
>>> >> > On Jul 26, 2016, at 8:21 AM, Jestin Ma <jestinwith.an.e@gmail.com>
>>> >> > wrote:
>>> >> >
>>> >> > Hello, when running spark jobs, I can access the master UI (port
>>> 8080
>>> >> > one) no problem. However, I'm confused as to how to access the
web
>>> UI to see
>>> >> > jobs/tasks/stages/etc.
>>> >> >
>>> >> > I can access the master UI at http://<master-node>:8080.
But port
>>> 4040
>>> >> > gives me a -connection cannot be reached-.
>>> >> >
>>> >> > Is the web UI http://<master-node> with a port of 4040?
>>> >> >
>>> >> > I'm running my Spark job on a cluster machine and submitting it
to a
>>> >> > master node part of the cluster. I heard of ssh tunneling; is that
>>> relevant
>>> >> > here?
>>> >> >
>>> >> > Thank you!
>>> >>
>>> >
>>>
>>
>>
>

Mime
View raw message