hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <tuc...@gmail.com>
Subject Re: Looking to a Hadoop 3 release
Date Thu, 05 Mar 2015 21:05:56 GMT
IMO, if part of the community wants to take on the responsibility and work
that takes to do a new major release, we should not discourage them from
doing that.

Having multiple major branches active is a standard practice.

This time around we are not replacing the guts as we did from Hadoop 1 to
Hadoop 2, but superficial surgery to address issues were not considered (or
was too much to take on top of the guts transplant).

For the split brain concern, we did a great of job maintaining Hadoop 1 and
Hadoop 2 until Hadoop 1 faded away.

Based on that experience I would say that the coexistence of Hadoop 2 and
Hadoop 3 will be much less demanding/traumatic.

Also, to facilitate the coexistence we should limit Java language features
to Java 7 (even if the runtime is Java 8), once Java 7 is not used anymore
we can remove this limitation.


On Thu, Mar 5, 2015 at 11:40 AM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:

> The 'resistance' is not so much about  a new major release, more so about
> the content and the roadmap of the release. Other than the two specific
> features raised (the need for breaking compat for them is something that I
> am debating), I haven't seen a roadmap of branch-3 about any more features
> that this community needs to discuss about. If all the difference between
> branch-2 and branch-3 is going to be JDK + a couple of incompat changes, it
> is a big problem in two dimensions (1) it's a burden keeping the branches
> in sync and avoiding the split-brain we experienced with 1.x, 2.x or worse
> branch-0.23, branch-2 and (2) very hard to ask people to not break more
> things in branch-3.
> We seem to have agreed upon a course of action for JDK7. And now we are
> taking a different direction for JDK8. Going by this new proposal, come
> 2016, we will have to deal with JDK9 and 3 mainline incompatible hadoop
> releases.
> Regarding, individual improvements like classpath isolation, shell script
> stuff, Jason Lowe captured it perfectly on HADOOP-11656 - it should be
> possible for every major feature that we develop to be a opt in, unless the
> change is so great and users can balance out the incompatibilities for the
> new stuff they are getting. Even with an ground breaking change like with
> YARN, we spent a bit of time to ensure compatibility (MAPREDUCE-5108) that
> has paid so many times over in return. Breaking compatibility shouldn't
> come across as too cheap a thing.
> Thanks,
> +Vinod
> On Mar 4, 2015, at 10:15 AM, Andrew Wang <andrew.wang@cloudera.com<mailto:
> andrew.wang@cloudera.com>> wrote:
> Where does this resistance to a new major release stem from? As I've
> described from the beginning, this will look basically like a 2.x release,
> except for the inclusion of classpath isolation by default and target
> version JDK8. I've expressed my desire to maintain API and wire
> compatibility, and we can audit the set of incompatible changes in trunk to
> ensure this. My proposal for doing alpha and beta releases leading up to GA
> also gives downstreams a nice amount of time for testing and validation.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message