spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From roni <roni.epi...@gmail.com>
Subject Re: Resource manager UI for Spark applications
Date Tue, 03 Mar 2015 22:09:34 GMT
Ted,
If the application is running then the logs are not available. Plus what i
want to view is the details about the running app as in spark UI.
Do I have to open some ports or do some other setting changes?




On Tue, Mar 3, 2015 at 10:08 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> bq. changing the address with internal to the external one , but still
> does not work.
> Not sure what happened.
> For the time being, you can use yarn command line to pull container log
> (put in your appId and container Id):
> yarn logs -applicationId application_1386639398517_0007 -containerId
> container_1386639398517_0007_01_000019
>
> Cheers
>
> On Tue, Mar 3, 2015 at 9:50 AM, roni <roni.epi112@gmail.com> wrote:
>
>> Hi Ted,
>>  I  used s3://support.elasticmapreduce/spark/install-spark to install
>> spark on my EMR cluster. It is 1.2.0.
>>  When I click on the link for history or logs it takes me to
>>
>> http://ip-172-31-43-116.us-west-2.compute.internal:9035/node/containerlogs/container_1424105590052_0070_01_000001/hadoop
>>  and I get -
>>
>> The server at *ip-172-31-43-116.us-west-2.compute.internal* can't be
>> found, because the DNS lookup failed. DNS is the network service that
>> translates a website's name to its Internet address. This error is most
>> often caused by having no connection to the Internet or a misconfigured
>> network. It can also be caused by an unresponsive DNS server or a firewall
>> preventing Google Chrome from accessing the network.
>> I tried  changing the address with internal to the external one , but
>> still does not work.
>> Thanks
>> _roni
>>
>>
>> On Tue, Mar 3, 2015 at 9:05 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>>> bq. spark UI does not work for Yarn-cluster.
>>>
>>> Can you be a bit more specific on the error(s) you saw ?
>>>
>>> What Spark release are you using ?
>>>
>>> Cheers
>>>
>>> On Tue, Mar 3, 2015 at 8:53 AM, Rohini joshi <roni.epi112@gmail.com>
>>> wrote:
>>>
>>>> Sorry , for half email - here it is again in full
>>>> Hi ,
>>>> I have 2 questions -
>>>>
>>>>  1. I was trying to use Resource Manager UI for my SPARK application
>>>> using yarn cluster mode as I observed that spark UI does not work for
>>>> Yarn-cluster.
>>>> IS that correct or am I missing some setup?
>>>>
>>>> 2. when I click on Application Monitoring or history , i get
>>>> re-directed to some linked with internal Ip address. Even if I replace that
>>>> address with the public IP , it still does not work.  What kind of setup
>>>> changes are needed for that?
>>>>
>>>> Thanks
>>>> -roni
>>>>
>>>> On Tue, Mar 3, 2015 at 8:45 AM, Rohini joshi <roni.epi112@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi ,
>>>>> I have 2 questions -
>>>>>
>>>>>  1. I was trying to use Resource Manager UI for my SPARK application
>>>>> using yarn cluster mode as I observed that spark UI does not work for
>>>>> Yarn-cluster.
>>>>> IS that correct or am I missing some setup?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message