spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From roni <roni.epi...@gmail.com>
Subject Re: Resource manager UI for Spark applications
Date Tue, 03 Mar 2015 22:17:26 GMT
ah!! I think I know what you mean. My job was just in "accepted" stage for
a long time as it was running a huge file.
But now that it is in running stage , I can see it . I can see it at post
9046 though instead of 4040 . But  I can see it.
Thanks
-roni

On Tue, Mar 3, 2015 at 1:19 PM, Zhan Zhang <zzhang@hortonworks.com> wrote:

>  In Yarn (Cluster or client), you can access the spark ui when the app is
> running. After app is done, you can still access it, but need some extra
> setup for history server.
>
>  Thanks.
>
>  Zhan Zhang
>
>  On Mar 3, 2015, at 10:08 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
>  bq. changing the address with internal to the external one , but still
> does not work.
> Not sure what happened.
> For the time being, you can use yarn command line to pull container log
> (put in your appId and container Id):
>  yarn logs -applicationId application_1386639398517_0007 -containerId
> container_1386639398517_0007_01_000019
>
>  Cheers
>
> On Tue, Mar 3, 2015 at 9:50 AM, roni <roni.epi112@gmail.com> wrote:
>
>>    Hi Ted,
>>   I  used s3://support.elasticmapreduce/spark/install-spark to install
>> spark on my EMR cluster. It is 1.2.0.
>>   When I click on the link for history or logs it takes me to
>>
>> http://ip-172-31-43-116.us-west-2.compute.internal:9035/node/containerlogs/container_1424105590052_0070_01_000001/hadoop
>>   and I get -
>>
>> The server at *ip-172-31-43-116.us
>> <http://ip-172-31-43-116.us>-west-2.compute.internal* can't be found,
>> because the DNS lookup failed. DNS is the network service that translates a
>> website's name to its Internet address. This error is most often caused by
>> having no connection to the Internet or a misconfigured network. It can
>> also be caused by an unresponsive DNS server or a firewall preventing Google
>> Chrome from accessing the network.
>>  I tried  changing the address with internal to the external one , but
>> still does not work.
>>  Thanks
>>  _roni
>>
>>
>> On Tue, Mar 3, 2015 at 9:05 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>>> bq. spark UI does not work for Yarn-cluster.
>>>
>>>  Can you be a bit more specific on the error(s) you saw ?
>>>
>>>  What Spark release are you using ?
>>>
>>>  Cheers
>>>
>>> On Tue, Mar 3, 2015 at 8:53 AM, Rohini joshi <roni.epi112@gmail.com>
>>> wrote:
>>>
>>>>   Sorry , for half email - here it is again in full
>>>>  Hi ,
>>>>  I have 2 questions -
>>>>
>>>>   1. I was trying to use Resource Manager UI for my SPARK application
>>>> using yarn cluster mode as I observed that spark UI does not work for
>>>> Yarn-cluster.
>>>>  IS that correct or am I missing some setup?
>>>>
>>>>  2. when I click on Application Monitoring or history , i get
>>>> re-directed to some linked with internal Ip address. Even if I replace that
>>>> address with the public IP , it still does not work.  What kind of setup
>>>> changes are needed for that?
>>>>
>>>>  Thanks
>>>>  -roni
>>>>
>>>> On Tue, Mar 3, 2015 at 8:45 AM, Rohini joshi <roni.epi112@gmail.com>
>>>> wrote:
>>>>
>>>>>  Hi ,
>>>>>  I have 2 questions -
>>>>>
>>>>>   1. I was trying to use Resource Manager UI for my SPARK application
>>>>> using yarn cluster mode as I observed that spark UI does not work for
>>>>> Yarn-cluster.
>>>>>  IS that correct or am I missing some setup?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
>

Mime
View raw message