spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhan Zhang (JIRA)" <>
Subject [jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server
Date Sat, 21 Feb 2015 01:11:12 GMT


Zhan Zhang commented on SPARK-1537:

[~vanzin]  We should centralized all comments and reviews in one place, instead of going to
different links. Also, we want to the reviewed code is updated, instead of based on some old

Let's go to technical:

1. We all agree on this one about timeline client, and this is why it is alpha feature. Hive
is a good example, but nobody can deny its importance in spark.
2. ACL is included in the patch, but not in the spec.
3. I understand your question, but the scope of my respond may be too big. To solve this,
more work is needed on the entity design.

Let's keep an eye on these issues. 

> Add integration with Yarn's Application Timeline Server
> -------------------------------------------------------
>                 Key: SPARK-1537
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: YARN
>            Reporter: Marcelo Vanzin
>            Assignee: Marcelo Vanzin
>         Attachments: SPARK-1537.txt, spark-1573.patch
> It would be nice to have Spark integrate with Yarn's Application Timeline Server (see
YARN-321, YARN-1530). This would allow users running Spark on Yarn to have a single place
to go for all their history needs, and avoid having to manage a separate service (Spark's
built-in server).
> At the moment, there's a working version of the ATS in the Hadoop 2.4 branch, although
there is still some ongoing work. But the basics are there, and I wouldn't expect them to
change (much) at this point.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message