spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject Re: Spark UI and running spark-submit with --master yarn
Date Tue, 03 Mar 2015 00:47:20 GMT
That does not look like the RM UI. Please check your configuration for
the port (see Ted's e-mail).

On Mon, Mar 2, 2015 at 4:45 PM, Anupama Joshi <anupama.joshi@gmail.com> wrote:
> Hi ,
>  port 8088 does not show me anything .(can not connect)
> where as port ec2-52-10-234-111.us-west-2.compute.amazonaws.com:9026 shows
> me all the applications.
> Do I have to do anything for the port 8088 or whatever I am seeing at 9026
> port is good .Attached is screenshot .
> Thanks
> AJ
>
> On Mon, Mar 2, 2015 at 4:24 PM, Marcelo Vanzin <vanzin@cloudera.com> wrote:
>>
>> That's the RM's RPC port, not the web UI port. (See Ted's e-mail -
>> normally web UI is on 8088.)
>>
>> On Mon, Mar 2, 2015 at 4:14 PM, Anupama Joshi <anupama.joshi@gmail.com>
>> wrote:
>> > Hi Marcelo,
>> > Thanks for the quick reply.
>> > I have a EMR cluster and I am running the spark-submit on the master
>> > node in
>> > the cluster.
>> > When I start the spark-submit , I see
>> > 15/03/02 23:48:33 INFO client.RMProxy: Connecting to ResourceManager at
>> > /172.31.43.254:9022
>> > But If I try that URL or the use the external DNS
>> > ec2-52-10-234-111.us-west-2.compute.amazonaws.com:9022
>> > it does not work
>> > What am I missing here ?
>> > Thanks a lot for the help
>> > -AJ
>> >
>> >
>> > On Mon, Mar 2, 2015 at 3:50 PM, Marcelo Vanzin <vanzin@cloudera.com>
>> > wrote:
>> >>
>> >> What are you calling "<masternode>"? In yarn-cluster mode, the driver
>> >> is running somewhere in your cluster, not on the machine where you run
>> >> spark-submit.
>> >>
>> >> The easiest way to get to the Spark UI when using Yarn is to use the
>> >> Yarn RM's web UI. That will give you a link to the application's UI
>> >> regardless of whether it's running on client or cluster mode.
>> >>
>> >> On Mon, Mar 2, 2015 at 3:39 PM, Anupama Joshi <anupama.joshi@gmail.com>
>> >> wrote:
>> >> > Hi ,
>> >> >
>> >> >  When I run my application with --master yarn-cluster or --master
>> >> > yarn
>> >> > --deploy-mode cluster , I can not  the spark UI at the  location --
>> >> > <masternode>:4040Even if I am running the job , I can not see
teh
>> >> > SPARK
>> >> > UI.
>> >> > When I run with --master yarn --deploy-mode client  -- I see the
>> >> > Spark
>> >> > UI
>> >> > but I cannot see my job  running.
>> >> >
>> >> > When I run spark-submit with --master local[*] , I see the spark UI
,
>> >> > my
>> >> > job
>> >> > everything (Thats great)
>> >> >
>> >> > Do I need to do some settings to see the UI?
>> >> >
>> >> > Thanks
>> >> >
>> >> > -AJ
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Marcelo
>> >
>> >
>>
>>
>>
>> --
>> Marcelo
>
>



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message