hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wang <andrew.w...@cloudera.com>
Subject Re: Branch merges and 3.0.0-beta1 scope
Date Fri, 25 Aug 2017 22:29:16 GMT
Here's a summary of some 1-on-1 conversations I had with contributors of
the different features I'm tracking.

Storage Policy Satisfier (Uma Gangumalla)
* Target version: 3.1.0, maybe 3.0.0 GA
* We're resolving some design discussions on JIRA. Plan is to do some MVP
work on the API to get this into 3.1, and if we're happy with the second
phase, consider for 3.0 GA.

YARN native services (Jian He)
* Target version: 3.0.0-beta1 as an alpha feature
* I was convinced that this is very separate from the core. I'll get
someone from Cloudera to run it through our integration tests to verify it
doesn't break anything downstream, then happy to merge.

Resource profiles (Wangda Tan)
* Target version: 3.0.0 GA
* Already provided update above, we're going to test it a lot and target
for GA.

HDFS router-based federation (Chris Douglas)
* Target version: 3.0.0 GA
* This is like YARN federation, very separate and doesn't add new APIs, run
in production.
* If it passes our internal integration testing, I'm fine putting this in

HDFS tiered storage (Chris Douglas):
* Target version: 3.1.0
* This touches some core stuff, and the write path is still being worked
on. Still somewhat useful with just the read path. Targeting at 3.1.0 gives
enough time to wrap this up.

TSv2 phase 2 (Vrushali C)
* Target version: 3.0.0-beta1
* This is more like "beta" in terms of readiness, Twitter is planning to
roll it out to production.
* I double checked with Haibo, and he successfully ran it through our
internal integration testing.

Thanks to everyone for meeting with me on short notice, and being very
reasonable about target versions and quality bars. If I mischaracterized
any of our discussions, please reach out or comment.

The branching and versioning discussion is still proceeding. I'd ask that
the pending merge VOTEs watch this carefully; I'm hoping to resolve the
discussion and branch before the VOTEs close, but let's make sure the
branches and versions are ready before doing the actual merges.


On Fri, Aug 25, 2017 at 11:06 AM, Wangda Tan <wheeleast@gmail.com> wrote:

> Resource profile is similar to TSv2, the feature is:
> - Alpha feature, we will not freeze new added APIs. And all added APIs are
> explicitly marked to @Unstable.
> - Allow rolling upgrade from branch-2.
> - Touched existing code, but we have, and will continue tests to make sure
> changes are safe.
> Discussed with Andrew offline, we decided to not put this to beta1 since
> beta1 is not far away. But we want to put it before GA if sufficient tests
> are done.
> Thanks,
> Wangda
> On Fri, Aug 25, 2017 at 10:54 AM, Rohith Sharma K S <
> rohithsharmaks@apache.org> wrote:
>> On 25 August 2017 at 22:39, Andrew Wang <andrew.wang@cloudera.com> wrote:
>> > Hi Rohith,
>> >
>> > Given that we're advertising TSv2 as an alpha feature, I think we're
>> > allowed to break compatibility. Let's make sure this is clear in the
>> > release notes and documentation.
>> >
>> > That said, with TSv2 phase 2, is the API going to be frozen? The
>> umbrella
>> > JIRA refers to "TSv2 alpha2" which indicated to me it was still
>> alpha-level
>> > quality and stability.
>> >
>> YES, We have decided to freeze API's. I do not think we make any
>> compatibility break in future.
>> >
>> > Best,
>> > Andrew
>> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message