hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vrushali Channapattan <vrushalic2...@gmail.com>
Subject [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk
Date Tue, 22 Aug 2017 06:32:49 GMT
Hi folks,

Per earlier discussion [1], I'd like to start a formal vote to merge
feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. The vote will
run for 7 days, and will end August 29 11:00 PM PDT.

We have previously completed one merge onto trunk [3] and Timeline Service
v2 has been part of Hadoop release 3.0.0-alpha1.

Since then, we have been working on extending the capabilities of Timeline
Service v2 in a feature branch [2] for a while, and we are reasonably
confident that the state of the feature meets the criteria to be merged
onto trunk and we'd love folks to get their hands on it in a test capacity
and provide valuable feedback so that we can make it production-ready.

In a nutshell, Timeline Service v.2 delivers significant scalability and
usability improvements based on a new architecture. What we would like to
merge to trunk is termed "alpha 2" (milestone 2). The feature has a
complete end-to-end read/write flow with security and read level
authorization via whitelists. You should be able to start setting it up and
testing it.

At a high level, the following are the key features that have been
implemented since alpha1:
- Security via Kerberos Authentication and delegation tokens
- Read side simple authorization via whitelist
- Client configurable entity sort ordering
- Richer REST APIs for apps, app attempts, containers, fetching metrics by
timerange, pagination, sub-app entities
- Support for storing sub-application entities (entities that exist outside
the scope of an application)
- Configurable TTLs (time-to-live) for tables, configurable table prefixes,
configurable hbase cluster
- Flow level aggregations done as dynamic (table level) coprocessors
- Uses latest stable HBase release 1.2.6

There are a total of 82 subtasks that were completed as part of this effort.

We paid close attention to ensure that once disabled Timeline Service v.2
does not impact existing functionality when disabled (by default).

Special thanks to a team of folks who worked hard and contributed towards
this effort with patches, reviews and guidance: Rohith Sharma K S, Varun
Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep
Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack.


[1] http://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg27383.html
[2] https://issues.apache.org/jira/browse/YARN-5355
[3] https://issues.apache.org/jira/browse/YARN-2928
[4] https://github.com/apache/hadoop/commits/YARN-5355

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message