spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ruebenacker, Oliver A" <Oliver.Ruebenac...@altisource.com>
Subject RE: Web UI
Date Thu, 04 Sep 2014 12:28:02 GMT

     Hello,

  Thanks for the link – this is for standalone, though, and most URLs don’t work for local.
  I will look into deploying as standalone on a single node for testing and development.

     Best, Oliver

From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: Thursday, September 04, 2014 3:09 AM
To: Ruebenacker, Oliver A
Cc: Wonha Ryu; user@spark.apache.org
Subject: Re: Web UI

Hi

You can see this doc<https://spark.apache.org/docs/latest/spark-standalone.html#configuring-ports-for-network-security>
for all the available webUI ports.

Yes there are ways to get the data metrics in Json format, One of them is below:

​​
http://webUI:8080/json/  Or simply
​​
curl webUI:8080/json/

There are some PRs about it you can read it over here https://github.com/apache/spark/pull/1682

Thanks
Best Regards

On Thu, Sep 4, 2014 at 2:24 AM, Ruebenacker, Oliver A <Oliver.Ruebenacker@altisource.com<mailto:Oliver.Ruebenacker@altisource.com>>
wrote:

     Hello,

  Interestingly, http://localhost:4040/metrics/json/ gives some numbers, but only a few which
never seem to change during the application’s lifetime.

  Either the web UI has some very strange limitations, or there are some URLs yet to be discovered
that do something interesting.

     Best,
     Oliver


From: Wonha Ryu [mailto:wonha.ryu@gmail.com<mailto:wonha.ryu@gmail.com>]
Sent: Wednesday, September 03, 2014 4:27 PM

To: Ruebenacker, Oliver A
Cc: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: Web UI

Hey Oliver,

IIRC there's no JSON endpoint for application web UI. They only exist for cluster master and
worker.

- Wonha


On Wed, Sep 3, 2014 at 12:58 PM, Ruebenacker, Oliver A <Oliver.Ruebenacker@altisource.com<mailto:Oliver.Ruebenacker@altisource.com>>
wrote:

     Hello,

  Thanks for the help! But I tried starting with “–master local[4]” and when I load
http://localhost:4040/json I just get forwarded to http://localhost:4040/stages/, and it’s
all human-readable HTML, no JSON.

     Best,
     Oliver


From: Wonha Ryu [mailto:wonha.ryu@gmail.com<mailto:wonha.ryu@gmail.com>]
Sent: Wednesday, September 03, 2014 3:36 PM
To: Ruebenacker, Oliver A
Cc: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: Web UI

Hi Oliver,

Spark standalone master and worker support '/json' endpoint in web UI, which returns some
of the information in JSON format.
I wasn't able to find relevant documentation, though.

- Wonha

On Wed, Sep 3, 2014 at 12:12 PM, Ruebenacker, Oliver A <Oliver.Ruebenacker@altisource.com<mailto:Oliver.Ruebenacker@altisource.com>>
wrote:

     Hello,

  What is included in the Spark web UI? What are the available URLs? Can the information be
obtained in a machine-readable way (e.g. JSON, XML, etc)?

  Thanks!

     Best,
     Oliver

Oliver Ruebenacker | Solutions Architect

Altisource™
290 Congress St, 7th Floor | Boston, Massachusetts 02210
P: (617) 728-5582<tel:%28617%29%20728-5582> | ext: 275585
Oliver.Ruebenacker@Altisource.com<mailto:Oliver.Ruebenacker@Altisource.com> | www.Altisource.com<http://www.Altisource.com>

***********************************************************************************************************************

This email message and any attachments are intended solely for the use of the addressee. If
you are not the intended recipient, you are prohibited from reading, disclosing, reproducing,
distributing, disseminating or otherwise using this transmission. If you have received this
message in error, please promptly notify the sender by reply email and immediately delete
this message from your system.
This message and any attachments may contain information that is confidential, privileged
or exempt from disclosure. Delivery of this message to any person other than the intended
recipient is not intended to waive any right or privilege. Message transmission is not guaranteed
to be secure or free of software viruses.
***********************************************************************************************************************

***********************************************************************************************************************

This email message and any attachments are intended solely for the use of the addressee. If
you are not the intended recipient, you are prohibited from reading, disclosing, reproducing,
distributing, disseminating or otherwise using this transmission. If you have received this
message in error, please promptly notify the sender by reply email and immediately delete
this message from your system.
This message and any attachments may contain information that is confidential, privileged
or exempt from disclosure. Delivery of this message to any person other than the intended
recipient is not intended to waive any right or privilege. Message transmission is not guaranteed
to be secure or free of software viruses.
***********************************************************************************************************************

***********************************************************************************************************************

This email message and any attachments are intended solely for the use of the addressee. If
you are not the intended recipient, you are prohibited from reading, disclosing, reproducing,
distributing, disseminating or otherwise using this transmission. If you have received this
message in error, please promptly notify the sender by reply email and immediately delete
this message from your system.
This message and any attachments may contain information that is confidential, privileged
or exempt from disclosure. Delivery of this message to any person other than the intended
recipient is not intended to waive any right or privilege. Message transmission is not guaranteed
to be secure or free of software viruses.
***********************************************************************************************************************

***********************************************************************************************************************

This email message and any attachments are intended solely for the use of the addressee. If
you are not the intended recipient, you are prohibited from reading, disclosing, reproducing,
distributing, disseminating or otherwise using this transmission. If you have received this
message in error, please promptly notify the sender by reply email and immediately delete
this message from your system. This message and any attachments may contain information that
is confidential, privileged or exempt from disclosure. Delivery of this message to any person
other than the intended recipient is not intended to waive any right or privilege. Message
transmission is not guaranteed to be secure or free of software viruses.
***********************************************************************************************************************
Mime
View raw message