hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Kambatla <ka...@cloudera.com>
Subject Re: [VOTE] Release cadence and EOL
Date Sun, 22 Jan 2017 03:08:54 GMT
Given the discussions, I feel we are not ready for VOTE on this yet.
Sangjin, should we go back to the DISCUSS thread?

IMO, these are guidelines and not policies we want to enforce. Maybe, the
text should say something along the lines of: "The Hadoop community is
inclined to". And, maybe, a caveat that these inclinations will be acted
upon based on the release in question, its adoption and the availability of
committer volunteers.

As I said on the DISCUSS thread, IMO, these guidelines are very useful and
form a good baseline to operate from as Sangjin said. Also, I don't see any
disadvantages of having this baseline.

Here are the advantages I see.

   1. Users know what to expect in terms of security fixes, critical bug
   fixes and can plan their upgrades accordingly.
   2. Contributors don't need to panic about getting a feature/improvement
   in *this* release, because they know the next one is only 6 months away.
   3. RM: some method to madness. Junping, for instance, is trying to roll
   a release with 2300 patches. It is a huge time investment. (Thanks again,
   Junping.) Smaller releases are easier to manage. A target release cadence,
   coupled with a process that encourages volunteering, IMO would lead to more
   committers doing releases.

To conclude, the biggest value I see is us (the community) agreeing on good
practices for our releases and work towards that. Writing it down somewhere
makes it a little more formal like the compatibility stuff, even if it is
not enforceable.

On Sat, Jan 21, 2017 at 11:36 AM, Chris Douglas <cdouglas@apache.org> wrote:

> On Fri, Jan 20, 2017 at 2:50 PM, Sangjin Lee <sjlee@apache.org> wrote:
> > The security patch for the 2.6.x line is a case in point. Without any
> > guideline, we would start with "What should we do for 2.6.x? Should we
> > continue to patch it?" With this guideline, the baseline is already "it's
> > been 2 years since 2.6.0 is released and we should consider stopping
> > releasing from 2.6.x and encourage users to upgrade to 2.7.x."
>
> Unless it falls under the "good reason" clause. To invent an example,
> if 3.6.x were within the two year/minor release window, but 3.5.x was
> more widely deployed/stable, then we'd use this escape hatch to patch
> 3.5.x and likely just violate our policy on 3.6.x (when the
> implementation cost is high). How do we "force" a fix to 3.6.x?
>
> We can't actually compel work from people. Even when we can point to a
> prior consensus, someone needs to be motivated to actually complete
> that task. That's the rub: this proposal doesn't only allow us to stop
> working on old code, it also promises that we'll work on code we want
> to abandon.
>
> Pointing to a bylaw, and telling a contributor they "must" support a
> branch/release isn't going to result in shorter discussions, either.
> In the preceding hypothetical, if someone wants a fix in the 3.6 line,
> they either need to convince others that it's important or they need
> to do the work themselves.
>
> > Actually the decision on security issues is a pretty strong indicator of
> our
> > desire for EOL. If we say we do not want to patch that line for security
> > vulnerability, then there would be even *less* rationale for fixing any
> > other issue on that line. So the decision to stop backporting security
> > patches is a sufficient indication of EOL in my mind.
>
> Agreed. If we're not backporting security patches to a branch, then we
> need to release a CVE, file a JIRA, and move on. If someone wants to
> fix it and roll an RC for that release line, it lives. Otherwise it
> dies as people move to later versions (unpatched security flaws are
> motivating). A branch is EOL when we stop releasing from it. Two years
> or two minor releases is a good heuristic based on recent history, but
> overfitting policy to experience doesn't seem to buy us anything.
>
> I'm all for spending less time discussing release criteria, but if
> it's sufficient to observe which release lines are getting updates and
> label them accordingly, that's cheaper to implement than a curated set
> of constraints. -C
>
> >> We can still embargo security flaws if someone asks (to give them time
> >> time to implement a fix and call a vote). If there's nothing else in
> >> the release, then we're effectively announcing it. In those cases, we
> >> call a vote on private@ (cc: security@). -C
> >>
> >>
> >> On Thu, Jan 19, 2017 at 1:30 PM, Andrew Wang <andrew.wang@cloudera.com>
> >> wrote:
> >> > I don't think the motivation here is vendor play or taking away power
> >> > from
> >> > committers. Having a regular release cadence helps our users
> understand
> >> > when a feature will ship so they can plan their upgrades. Having an
> EOL
> >> > policy and a minimum support period helps users choose a release line,
> >> > and
> >> > understand when they will need to upgrade.
> >> >
> >> > In the earlier thread, we discussed how these are not rules, but
> >> > guidelines. There's a lot of flexibility if someone wants to keep
> >> > maintaining a release line (particularly if they are willing to do the
> >> > backporting work). More power to them; more releases are a good thing
> >> > for
> >> > the project.
> >> >
> >> > My main concern (which I raised on the earlier thread) is that without
> >> > significant improvements to the release process and upstream
> integration
> >> > testing, it's unlikely we'll actually ship more releases. Too often,
> >> > branches are simply not in a releaseable state, or they have latent
> >> > blocker
> >> > bugs due to a lack of testing. This is what we've been struggling with
> >> > on
> >> > both the 2.8.x and 3.0.0-x release lines.
> >> >
> >> > So, in the abstract, I'm +1 on having a published policy on release
> >> > cadence
> >> > and EOL. This information helps users.
> >> >
> >> > However, I don't think we're ready to actually execute on this policy
> >> > for
> >> > the above reasons. This leaves me ambivalent overall, perhaps -0 since
> >> > publishing a policy we don't follow is more confusing to users.
> >> >
> >> > My 2c,
> >> > Andrew
> >> >
> >> >
> >> >
> >> > On Thu, Jan 19, 2017 at 12:28 PM, Arpit Agarwal
> >> > <aagarwal@hortonworks.com>
> >> > wrote:
> >> >
> >> >> The ASF release policy says releases may not be vetoed [1] so the EOL
> >> >> policy sounds unenforceable. Not sure a release cadence is
> enforceable
> >> >> either since Release Managers are volunteers.
> >> >>
> >> >> 1. https://www.apache.org/dev/release.html#approving-a-release
> >> >>
> >> >>
> >> >>
> >> >> On 1/18/17, 7:06 PM, "Junping Du" <jdu@hortonworks.com> wrote:
> >> >>
> >> >>     +1 on Sangjin's proposal -
> >> >>     "A minor release line is end-of-lifed 2 years after it is
> released
> >> >> or
> >> >> there
> >> >>     are 2 newer minor releases, whichever is sooner. The community
> >> >> reserves the
> >> >>     right to extend or shorten the life of a release line if there
> is a
> >> >> good
> >> >>     reason to do so."
> >> >>
> >> >>     I also noticed Karthik bring up some new proposals - some of them
> >> >> looks interesting to me and I have some ideas as well. Karthik, can
> you
> >> >> bring it out in a separated discussion threads so that we can discuss
> >> >> from
> >> >> there?
> >> >>
> >> >>     About Chris Trezzo's question about definition of EOL of hadoop
> >> >> release, I think potentially changes could be:
> >> >>     1. For users of Apache hadoop, they would expect to upgrade to
a
> >> >> new
> >> >> minor/major releases after EOL of their current release because there
> >> >> is no
> >> >> guarantee of new maintenance release.
> >> >>
> >> >>     2. For release effort, apache law claim that committer can
> >> >> volunteer
> >> >> RM for any release. With this release EOL proposal passes and written
> >> >> into
> >> >> hadoop bylaw, anyone want to call for a release which is EOL then
> >> >> she/he
> >> >> have to provide a good reason to community and get voted before to
> >> >> start
> >> >> release effort. We don't want to waste community time/resource to
> >> >> verify/vote a narrow interested release.
> >> >>
> >> >>     3. About committer's responsibility, I think the bottom line is
> >> >> committer should commit patch contributor's target release and
> her/his
> >> >> own
> >> >> interest release which I conservatively agree with Allen's point that
> >> >> this
> >> >> vote doesn't change anything. But if a committer want to take care
> more
> >> >> interest from the whole community like most committers are doing
> today,
> >> >> he/she should understand which branches can benefit more people and
> >> >> could
> >> >> skip some EOL release branches for backport effort.
> >> >>
> >> >>     About major release EOL, this could be more complicated and I
> think
> >> >> we
> >> >> should discuss separately.
> >> >>
> >> >>     Thanks,
> >> >>
> >> >>     Junping
> >> >>     ________________________________________
> >> >>     From: Allen Wittenauer <aw@effectivemachines.com>
> >> >>     Sent: Wednesday, January 18, 2017 3:30 PM
> >> >>     To: Chris Trezzo
> >> >>     Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> >>     Subject: Re: [VOTE] Release cadence and EOL
> >> >>
> >> >>     > On Jan 18, 2017, at 11:21 AM, Chris Trezzo <ctrezzo@gmail.com>
> >> >> wrote:
> >> >>     >
> >> >>     > Thanks Sangjin for pushing this forward! I have a few
> questions:
> >> >>
> >> >>             These are great questions, because I know I'm not seeing
> a
> >> >> whole lot of substance in this vote.  The way to EOL software in the
> >> >> open
> >> >> source universe is with new releases and aging it out.  If someone
> >> >> wants to
> >> >> be a RE for a new branch-1 release, more power to them.  As
> volunteers
> >> >> to
> >> >> the ASF, we're not on the hook to provide much actual support.  This
> >> >> feels
> >> >> more like a vendor play than a community one.  But if the PMC want
to
> >> >> vote
> >> >> on it, whatever.  It won't be first bylaw that doesn't really mean
> >> >> much.
> >> >>
> >> >>     > 1. What is the definition of end-of-life for a release in
the
> >> >> hadoop
> >> >>     > project? My current understanding is as follows: When a release
> >> >> line
> >> >>     > reaches end-of-life, there are no more planned releases for
> that
> >> >> line.
> >> >>     > Committers are no longer responsible for back-porting bug
fixes
> >> >> to
> >> >> the line
> >> >>     > (including fixed security vulnerabilities) and it is
> essentially
> >> >>     > unmaintained.
> >> >>
> >> >>             Just a point of clarification.  There is no policy that
> >> >> says
> >> >> that committers must back port.  It's up to the individual committers
> >> >> to
> >> >> push a change onto any particular branch. Therefore, this vote
> doesn't
> >> >> really change anything in terms of committer responsibilities here.
> >> >>
> >> >>     > 2. How do major releases affect the end-of-life proposal?
For
> >> >> example, how
> >> >>     > does a new minor release in the next major release affect
the
> >> >> end-of-life
> >> >>     > of minor releases in a previous major release? Is it possible
> to
> >> >> have a
> >> >>     > maintained 2.x release if there is a 3.3 release?
> >> >>
> >> >>             I'm looking forward to seeing this answer too, given that
> >> >> 2.7.0 is probably past the 2 year mark, 2.8.0 has seemingly been in
a
> >> >> holding pattern for over a year, and the next 3.0.0 alpha should be
> >> >> RSN....
> >> >>
> >> >>
> >> >> ------------------------------------------------------------
> ---------
> >> >>     To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >> >>     For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >> >>
> >> >>
> >> >>
> >> >> ------------------------------------------------------------
> ---------
> >> >>     To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> >>     For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message