hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Collins <...@cloudera.com>
Subject Re: Release numbering for branch-2 releases
Date Thu, 31 Jan 2013 00:21:15 GMT
Thanks for bringing this up Arun.  One of the issues is that we
haven't been clear about what type of compatibility breakages are
allowed, and which are not.  For example, renaming FileSystem#open is
incompatible, and not OK, regardless of the alpha/beta tag.  Breaking
a server/server APIs is OK pre-GA but probably not post GA, at least
in a point release, or required for a security fix, etc.
Configuration, data format, environment variable, changes etc can all
be similarly incompatible. The issue we had in HADOOP-9151 was someone
claimed it is not an incompatible change because it doesn't break API
compatibility even though it breaks wire compatibility. So let's be
clear about the types of incompatibility we are or are not permitting.
 For example, will it be OK to merge a change before 2.2.0-beta that
requires an HDFS metadata upgrade? Or breaks client server wire
compatibility?  I've been assuming that changing an API annotated
Public/Stable still requires multiple major releases (one to deprecate
and one to remove), does the alpha label change that? To some people
the "alpha", "beta" label implies instability in terms of
quality/features, while to others it means unstable APIs (and to some
both) so it would be good to spell that out. In short, agree that we
really need to figure out what changes are permitted in what releases,
and we should update the docs accordingly (there's a start here:

Note that the 2.0.0 alpha release vote thread was clear that we
thought were all in agreement that we'd like to keep client/server
compatible post 2.0 - and there was no push back. We pulled a number
of jiras into the 2.0 release explicitly so that we could preserve
client/server compatibility going forward.  Here's the relevant part
of the thread as a refresher: http://s.apache.org/gQ

"2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC
envelope in branch-2, but didn't make it into this rc. So, that would
mean that future alphas would not be protocol-compatible with this
alpha. Per a discussion a few weeks ago, I think we all were in
agreement that, if possible, we'd like all 2.x to be compatible for
client-server communication, at least (even if we don't support
cross-version for the intra-cluster protocols)"


On Tue, Jan 29, 2013 at 12:56 PM, Arun C Murthy <acm@hortonworks.com> wrote:
> Folks,
>  There has been some discussions about incompatible changes in the hadoop-2.x.x-alpha
releases on HADOOP-9070, HADOOP-9151, HADOOP-9192 and few other jiras. Frankly, I'm surprised
about some of them since the 'alpha' moniker was precisely to harden apis by changing them
if necessary, borne out by the fact that every  single release in hadoop-2 chain has had incompatible
changes. This happened since we were releasing early, moving fast and breaking things. Furthermore,
we'll have more in future as move towards stability of hadoop-2 similar to HDFS-4362, HDFS-4364
et al in HDFS and YARN-142 (api changes) for YARN.
>  So, rather than debate more, I had a brief chat with Suresh and Todd. Todd suggested
calling the next release as hadoop-2.1.0-alpha to indicate the incompatibility a little better.
This makes sense to me, as long as we are clear that we won't make any further *feature* releases
in hadoop-2.0.x series (obviously we might be forced to do security/bug-fix release).
>  Going forward, I'd like to start locking down apis/protocols for a 'beta' release. This
way we'll have one *final* opportunity post hadoop-2.1.0-alpha to make incompatible changes
if necessary and we can call it hadoop-2.2.0-beta.
>  Post hadoop-2.2.0-beta we *should* lock down and not allow incompatible changes. This
will allow us to go on to a hadoop-2.3.0 as a GA release. This forces us to do a real effort
on making sure we lock down for hadoop-2.2.0-beta.
>  In summary:
>  # I plan to now release hadoop-2.1.0-alpha (this week).
>  # We make a real effort to lock down apis/protocols and release hadoop-2.2.0-beta, say
in March.
>  # Post 'beta' release hadoop-2.3.0 as 'stable' sometime in May.
>  I'll start a separate thread on 'locking protocols' w.r.t client-protocols v/s internal
protocols (to facilitate rolling upgrades etc.), let's discuss this one separately.
>  Makes sense? Thoughts?
> thanks,
> Arun
> PS:  Between hadoop-2.2.0-beta and hadoop-2.3.0 we *might* be forced to make some incompatible
changes due to *unforeseen circumstances*, but no more gratuitous changes are allowed.

View raw message