hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jian He <...@hortonworks.com>
Subject Re: [DISCUSS] Merge yarn-native-services branch into trunk
Date Thu, 17 Aug 2017 20:03:16 GMT
Thanks Andrew for the comments. Answers below:

- There are no new APIs added in YARN/Hadoop core. In fact, all the new code are running outside
of existing system and they are optional and require users to explicitly opt in. The new system’s
own rest API is not stable and will be evolving.
- We have been running/testing a version of the entire system internally for quite a while.
- I’d like to see this in hadoop3-beta1. Of course, we’ll take responsibility of moving
fast and not block the potential timeline.


On Aug 16, 2017, at 10:29 AM, Andrew Wang <andrew.wang@cloudera.com<mailto:andrew.wang@cloudera.com>>

Hi Jian,

Hadoop 3.0.0-beta1 is planned for mid-September. If the plan is to merge in hopefully the
next two weeks, that's very, very close to the goal release date. We've already got a pile
of blockers and criticals to resolve before then.

Could you comment on testing and API stability for this branch? YARN Federation was run at
high scale and did not add new APIs, which provided a lot of confidence in the merge.

I'll also raise the option of cutting branch-3 or branch-3.0 for the 3.0.0 efforts, and targeting
this for 3.1.0.


On Tue, Aug 15, 2017 at 1:56 PM, Jian He <jhe@hortonworks.com<mailto:jhe@hortonworks.com>>
Hi All,
I would like to bring up the discussion of merging yarn-native-services branch into trunk
in a few weeks. There are a few issues left under YARN-5079 that are being actively worked
upon. As soon as they are resolved, we plan on start a vote hopefully in next 2 weeks. The
goal is to get this in for hadoop3 beta.

The major work in this branch include below umbrella jiras:
 - YARN-5079. A native YARN framework (ApplicationMaster) to migrate and orchestrate existing
services to YARN either docker or non-docker based.
 - YARN-4793. A Rest API server for user to deploy a service via a simple JSON spec
 - YARN-4757. Extending today's service registry with a simple DNS service to enable users
to discover services deployed on YARN
 - YARN-6419. UI support for native-services on the new YARN UI
All these new services are optional and have to be explicitly enabled.

Special thanks to a team of folks who worked hard towards this: Billie Rinaldi, Gour Saha,
Vinod Kumar Vavilapalli, Jonathan Maron, Rohith Sharma K S, Sunil G, Akhil PB. This effort
could not be possible without their ideas and hard work.

Please share your thoughts. Thanks.


To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:yarn-dev-unsubscribe@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:yarn-dev-help@hadoop.apache.org>

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message