spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: Web UI
Date Thu, 04 Sep 2014 07:09:15 GMT
Hi

You can see this doc
<https://spark.apache.org/docs/latest/spark-standalone.html#configuring-ports-for-network-security>
for all the available webUI ports.

Yes there are ways to get the data metrics in Json format, One of them is
below:

*​​http://webUI:8080/json/ <http://webUI:8080/json/>*  Or simply
> *​​curl webUI:8080/json/*


There are some PRs about it you can read it over here
https://github.com/apache/spark/pull/1682

Thanks
Best Regards


On Thu, Sep 4, 2014 at 2:24 AM, Ruebenacker, Oliver A <
Oliver.Ruebenacker@altisource.com> wrote:

>
>
>      Hello,
>
>
>
>   Interestingly, http://localhost:4040/metrics/json/ gives some numbers,
> but only a few which never seem to change during the application’s lifetime.
>
>
>
>   Either the web UI has some very strange limitations, or there are some
> URLs yet to be discovered that do something interesting.
>
>
>
>      Best,
>
>      Oliver
>
>
>
>
>
> *From:* Wonha Ryu [mailto:wonha.ryu@gmail.com]
> *Sent:* Wednesday, September 03, 2014 4:27 PM
>
> *To:* Ruebenacker, Oliver A
> *Cc:* user@spark.apache.org
> *Subject:* Re: Web UI
>
>
>
> Hey Oliver,
>
>
>
> IIRC there's no JSON endpoint for application web UI. They only exist for
> cluster master and worker.
>
>
>
> - Wonha
>
>
>
>
>
> On Wed, Sep 3, 2014 at 12:58 PM, Ruebenacker, Oliver A <
> Oliver.Ruebenacker@altisource.com> wrote:
>
>
>
>      Hello,
>
>
>
>   Thanks for the help! But I tried starting with “–master local[4]” and
> when I load http://localhost:4040/json I just get forwarded to
> http://localhost:4040/stages/, and it’s all human-readable HTML, no JSON.
>
>
>
>      Best,
>
>      Oliver
>
>
>
>
>
> *From:* Wonha Ryu [mailto:wonha.ryu@gmail.com]
> *Sent:* Wednesday, September 03, 2014 3:36 PM
> *To:* Ruebenacker, Oliver A
> *Cc:* user@spark.apache.org
> *Subject:* Re: Web UI
>
>
>
> Hi Oliver,
>
>
>
> Spark standalone master and worker support '/json' endpoint in web UI,
> which returns some of the information in JSON format.
>
> I wasn't able to find relevant documentation, though.
>
>
>
> - Wonha
>
>
>
> On Wed, Sep 3, 2014 at 12:12 PM, Ruebenacker, Oliver A <
> Oliver.Ruebenacker@altisource.com> wrote:
>
>
>
>      Hello,
>
>
>
>   What is included in the Spark web UI? What are the available URLs? Can
> the information be obtained in a machine-readable way (e.g. JSON, XML, etc)?
>
>
>
>   Thanks!
>
>
>
>      Best,
>
>      Oliver
>
>
>
> Oliver Ruebenacker | Solutions Architect
>
>
>
> Altisource™
>
> 290 Congress St, 7th Floor | Boston, Massachusetts 02210
>
> P: (617) 728-5582 | ext: 275585
>
> Oliver.Ruebenacker@Altisource.com | www.Altisource.com
>
>
>
>
> ***********************************************************************************************************************
>
> This email message and any attachments are intended solely for the use of
> the addressee. If you are not the intended recipient, you are prohibited
> from reading, disclosing, reproducing, distributing, disseminating or
> otherwise using this transmission. If you have received this message in
> error, please promptly notify the sender by reply email and immediately
> delete this message from your system.
>
> This message and any attachments may contain information that is
> confidential, privileged or exempt from disclosure. Delivery of this
> message to any person other than the intended recipient is not intended to
> waive any right or privilege. Message transmission is not guaranteed to be
> secure or free of software viruses.
>
> ***********************************************************************************************************************
>
>
>
>
> ***********************************************************************************************************************
>
> This email message and any attachments are intended solely for the use of
> the addressee. If you are not the intended recipient, you are prohibited
> from reading, disclosing, reproducing, distributing, disseminating or
> otherwise using this transmission. If you have received this message in
> error, please promptly notify the sender by reply email and immediately
> delete this message from your system.
>
> This message and any attachments may contain information that is
> confidential, privileged or exempt from disclosure. Delivery of this
> message to any person other than the intended recipient is not intended to
> waive any right or privilege. Message transmission is not guaranteed to be
> secure or free of software viruses.
>
> ***********************************************************************************************************************
>
>
>
>
> ***********************************************************************************************************************
>
> This email message and any attachments are intended solely for the use of
> the addressee. If you are not the intended recipient, you are prohibited
> from reading, disclosing, reproducing, distributing, disseminating or
> otherwise using this transmission. If you have received this message in
> error, please promptly notify the sender by reply email and immediately
> delete this message from your system.
> This message and any attachments may contain information that is
> confidential, privileged or exempt from disclosure. Delivery of this
> message to any person other than the intended recipient is not intended to
> waive any right or privilege. Message transmission is not guaranteed to be
> secure or free of software viruses.
>
> ***********************************************************************************************************************
>

Mime
View raw message