spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gordon Hamilton <open.go...@gmail.com>
Subject Re: JSON Endpoints
Date Wed, 21 Aug 2013 08:46:58 GMT
Hi Patrick,

I've got my calls working now.  This is exactly the kind of thing I was looking for.

Thank you for all your help.

Best regards,
Gordon



On 21 Aug 2013, at 00:44, Patrick Wendell <pwendell@gmail.com> wrote:

> Hey Gordon,
> 
> We don't have a specification of the API right now (in 0.7.X this is a
> somewhat experimental feature). Each call returns a set of (k, v) pairs and
> the list of keys is static, so you can just do a function call in the
> browser to see what it returns and get the schema. The JSON protocol code
> is here:
> 
> https://github.com/mesos/spark/blob/branch-0.7/core/src/main/scala/spark/deploy/JsonProtocol.scala
> 
> For 0.8 this will be more cleanly specified.
> 
> - Patrick
> 
> 
> On Tue, Aug 20, 2013 at 4:25 PM, Gordon Hamilton <
> gordon.hamilton@wandisco.com> wrote:
> 
>> Hi Patrick, that's great. I think I've possibly started a build from
>> branch-0.7 but have been looking at branch-0.8 code. I'll try the formats
>> you have described in the morning.
>> 
>> Is there any documentation that outlines all that I could expect to see
>> returned from the json calls?
>> 
>> Thanks,
>> Gordon
>> 
>> On Tuesday, August 20, 2013, Patrick Wendell wrote:
>> 
>>> In 0.7.X these are actually of the format /app?format=json. In 0.8.X they
>>> will be the format you described here "/app/json". Does that work for
>> you?
>>> 
>>> - Patrick
>>> 
>>> 
>>> On Tue, Aug 20, 2013 at 8:53 AM, Gordon Hamilton <
>>> gordon.hamilton@wandisco.com <javascript:;>> wrote:
>>> 
>>>> Hi guys,
>>>> 
>>>> I'm new to Spark, so apologies if I've missed something obvious.
>>>> 
>>>> I'm investigating the possibility of using Spark from an external UI,
>> and
>>>> see that Spark 0.7.0 introduced JSON endpoints into the UI which allows
>>>> Spark's state to be queried.  I see in MasterWebUI.scala that "/json"
>> and
>>>> "/app/json" map to renderJson functions, but when I hit the url's from
>> a
>>>> browser, I see "No service available for [/json]" and "No service
>>> available
>>>> for [/app/json]" respectively.
>>>> 
>>>> Is there any documentation on how the end points work, or could someone
>>>> point me in the right direction as to where I can find more
>> information?
>>>> 
>>>> Many thanks in advance,
>>>> Gordon
>>>> 
>>>> 
>>>> --
>>>> *Register now for* Subversion & Git Live October 2013<
>>>> http://www.wandisco.com/subversion-live-2013>
>>>> * **in** **Boston / San Francisco /** London*
>>>> 
>>>> THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY, AND MAY
>>> BE
>>>> PRIVILEGED.  If this message was misdirected, WANdisco, Inc. and its
>>>> subsidiaries, ("WANdisco") does not waive any confidentiality or
>>> privilege.
>>>> If you are not the intended recipient, please notify us immediately
>> and
>>>> destroy the message without disclosing its contents to anyone.  Any
>>>> distribution, use or copying of this e-mail or the information it
>>> contains
>>>> by other than an intended recipient is unauthorized.  The views and
>>>> opinions expressed in this e-mail message are the author's own and may
>>> not
>>>> reflect the views and opinions of WANdisco, unless the author is
>>> authorized
>>>> by WANdisco to express such views or opinions on its behalf.  All email
>>>> sent to or from this address is subject to electronic storage and
>> review
>>> by
>>>> WANdisco.  Although WANdisco operates anti-virus programs, it does not
>>>> accept responsibility for any damage whatsoever caused by viruses being
>>>> passed.
>>>> 
>>>> 
>>> 
>> 
>> 
>> --
>> Gordon Hamilton I *Hadoop Development Team*
>> 
>> WANdisco // *Non-Stop Data*
>> 
>> e. gordon.hamilton@wandisco.com
>> 
>> --
>> *Register now for* Subversion & Git Live October 2013<
>> http://www.wandisco.com/subversion-live-2013>
>> * **in** **Boston / San Francisco /** London*
>> 
>> THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY, AND MAY BE
>> PRIVILEGED.  If this message was misdirected, WANdisco, Inc. and its
>> subsidiaries, ("WANdisco") does not waive any confidentiality or privilege.
>> If you are not the intended recipient, please notify us immediately and
>> destroy the message without disclosing its contents to anyone.  Any
>> distribution, use or copying of this e-mail or the information it contains
>> by other than an intended recipient is unauthorized.  The views and
>> opinions expressed in this e-mail message are the author's own and may not
>> reflect the views and opinions of WANdisco, unless the author is authorized
>> by WANdisco to express such views or opinions on its behalf.  All email
>> sent to or from this address is subject to electronic storage and review by
>> WANdisco.  Although WANdisco operates anti-virus programs, it does not
>> accept responsibility for any damage whatsoever caused by viruses being
>> passed.
>> 
>> 



Mime
View raw message