spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vadim Semenov <vadim.seme...@datadoghq.com>
Subject Re: Spark REST API YARN client mode is not full?
Date Thu, 06 Oct 2016 17:14:11 GMT
It may be related to the CDH version of spark you're using.
When I use REST API I get yarn application id there

Try opening http://localhost:4040/api/v1/applications/0/stages

On Thu, Oct 6, 2016 at 8:40 AM, Vladimir Tretyakov <
vladimir.tretyakov@sematext.com> wrote:

> Hi,
>
> When I start Spark  v1.6 (cdh5.8.0) in Yarn client mode I see that 4040
> port is avaiable, but UI shows nothing and API returns not full
> information.
>
> I started Spark application like this:
>
> spark-submit     --master yarn-client     --class
> org.apache.spark.examples.SparkPi
> /usr/lib/spark/examples/lib/spark-examples-1.6.0-cdh5.8.0-
> hadoop2.6.0-cdh5.8.0.jar
>     10000
>
> API returns me:
>
> http://localhost:4040/api/v1/applications
>
> [ {
>   "name" : "Spark Pi",
>   "attempts" : [ {
>     "startTime" : "2016-10-05T11:27:54.558GMT",
>     "endTime" : "1969-12-31T23:59:59.999GMT",
>     "sparkUser" : "",
>     "completed" : false
>   } ]
> } ]
>
> Where is application id? How I can get more detailed information about
> application without this id (I am talking about
> /applications/[app-id]/jobs,
> /applications/[app-id]/stages etc urls from
> http://spark.apache.org/docs/1.6.0/monitoring.html)?
>
> UI also shows me empty pages.
>
> Without appId we cannot use other REST API calls. Is there any other way
> to get RUNNING application ids?
>
> Please help me understand what's going on.
>

Mime
View raw message