spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anupama Joshi <anupama.jo...@gmail.com>
Subject Re: Spark UI and running spark-submit with --master yarn
Date Tue, 03 Mar 2015 00:45:00 GMT
Hi ,
 port 8088 does not show me anything .(can not connect)
where as port ec2-52-10-234-111.us-west-2.compute.amazonaws.com:9026 shows
me all the applications.
Do I have to do anything for the port 8088 or whatever I am seeing at 9026
port is good .Attached is screenshot .
Thanks
AJ

On Mon, Mar 2, 2015 at 4:24 PM, Marcelo Vanzin <vanzin@cloudera.com> wrote:

> That's the RM's RPC port, not the web UI port. (See Ted's e-mail -
> normally web UI is on 8088.)
>
> On Mon, Mar 2, 2015 at 4:14 PM, Anupama Joshi <anupama.joshi@gmail.com>
> wrote:
> > Hi Marcelo,
> > Thanks for the quick reply.
> > I have a EMR cluster and I am running the spark-submit on the master
> node in
> > the cluster.
> > When I start the spark-submit , I see
> > 15/03/02 23:48:33 INFO client.RMProxy: Connecting to ResourceManager at
> > /172.31.43.254:9022
> > But If I try that URL or the use the external DNS
> > ec2-52-10-234-111.us-west-2.compute.amazonaws.com:9022
> > it does not work
> > What am I missing here ?
> > Thanks a lot for the help
> > -AJ
> >
> >
> > On Mon, Mar 2, 2015 at 3:50 PM, Marcelo Vanzin <vanzin@cloudera.com>
> wrote:
> >>
> >> What are you calling "<masternode>"? In yarn-cluster mode, the driver
> >> is running somewhere in your cluster, not on the machine where you run
> >> spark-submit.
> >>
> >> The easiest way to get to the Spark UI when using Yarn is to use the
> >> Yarn RM's web UI. That will give you a link to the application's UI
> >> regardless of whether it's running on client or cluster mode.
> >>
> >> On Mon, Mar 2, 2015 at 3:39 PM, Anupama Joshi <anupama.joshi@gmail.com>
> >> wrote:
> >> > Hi ,
> >> >
> >> >  When I run my application with --master yarn-cluster or --master yarn
> >> > --deploy-mode cluster , I can not  the spark UI at the  location --
> >> > <masternode>:4040Even if I am running the job , I can not see teh
> SPARK
> >> > UI.
> >> > When I run with --master yarn --deploy-mode client  -- I see the Spark
> >> > UI
> >> > but I cannot see my job  running.
> >> >
> >> > When I run spark-submit with --master local[*] , I see the spark UI ,
> my
> >> > job
> >> > everything (Thats great)
> >> >
> >> > Do I need to do some settings to see the UI?
> >> >
> >> > Thanks
> >> >
> >> > -AJ
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Marcelo
> >
> >
>
>
>
> --
> Marcelo
>

Mime
View raw message