hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer ...@altiscale.com>
Subject Re: In hindsight... Re: Thinking ahead to hadoop-2.6
Date Wed, 17 Sep 2014 16:55:23 GMT

On Sep 17, 2014, at 2:47 AM, Steve Loughran <stevel@hortonworks.com> wrote:
> I don't agree. Certainly the stuff I got into Hadoop 2.5 nailed down the
> filesystem binding with more tests than ever before.

	FWIW, based upon my survey of JIRA, there are a lot of unit test fixes that are only in trunk.

> But I am also aware of large organisations that are still on Java 6.
> Giving a clear roadmap "move to Java 7 now, java 8 in "XX months" can help
> them plan.

	Planning is a big thing.  That’s one of the reasons why it’d be prudent to start doing
3.0+JDK8 now as well.  Even if April slips, other projects and orgs are already moving to
8.  These people wonder what our plans are so that they can run one JVM.  Right now our answer
is ¯\_(ツ)_/¯ . 

	I’m sure I can dig up a user running Hadoop 0.13 because it ran on JDK5.  That doesn’t
mean the open source project should stall because certain orgs don’t/can't upgrade. 

>>        Drop the 2.6.0 release, branch trunk, and start rolling a
>> 3.0.0-alpha with JDK8 as the minimum.  2.5.1 becomes the base for all
>> sustaining work.  This gives the rest of the community time to move to JDK8
>> if they haven’t already.  For downstream vendors, it gives a roadmap for
>> their customers who will be asking about JDK8 sooner rather than later.  By
>> the time 3.0 stabilizes, we’re probably looking at April, which is perfect
>> timing.
> That delays getting stuff out too much; if april slips it becomes a long
> time since an ASF release came out.

	I’m assuming you specifically mean a ‘stable’ release.  If, as everyone seems to be
saying, that 3.x doesn’t have that much different than 2.x, doesn’t this mean that 3.x
should be stable much quicker than 2.x took?  In other words, if 2.5 is stable and the biggest
differences between it and trunk is the majority of code (450+ JIRAs as of yesterday afternoon)
that just also happens to be in 2.6, doesn’t it mean 2.6 is also extremely unstable?  (Thus
supporting my conjecture that 2.6 is going to be a problematic release?)

> Saying "you must run on Java 8 for
> this" will only scare people off and hold back adoption of 3.x, leaving 2.5
> as the last release that ends up being used for a while; the new 1.0.4

	From the outside, trunk looks a lot of 0.21 already.  From what I can tell, there is zero
motivation to get it out the door and on a roadmap. Primarily because there is little different
between trunk and branch-2.  This is a very dangerous place to be as those few differences,
some measured in years old, rot and wither. :(

> Here's an alternative
> -2.6 on java 6, announce EOL for Java 6 support
> -2.7 on Java 7, state that the lifespan of j7 support will be some bounded
> time period (12-18 mo)
> -trunk to build and test on Java 8 in jenkins alongside java 7. For that to
> be useful someone needs to volunteer to care about build failures. are you
> volunteering, Allen?

	This seems reasonable, except what release should folks who *require* java 8 use? Nightly
trunk+patches builds? How do downstream projects test?  Should JDK8 fixes be going into a
branch?  (I’m making the assumption that fixes for JDK8 are not backward compatible with
JDK7.  Hopefully they are, but given our usage of private APIs…)

	I’ve been approached by a few people over the past month+ if I’d be interested in or
will be RM’ing 3.0.  I’m seriously considering it esp given a) it’d be a nice learning
experience for me  b) my “day job” makes it practical time-wise c) I seem to be the only
one concerned enough about quite a bit of stale code  to get it out the door.

	FWIW, I’m in the process of moving my test vm to JDK8 to see how bad the damage truly is
right now. Based on others, it seems security doesn’t work, which is a pretty big deal.
 I can certainly start posting trunk builds on people.apache.org if folks are interested.

> -we switch trunk to Java 7 NOW. That doesn't mean a rewrite fest going
> through all catch() statements making them multicatch, and the same for
> string case.

	Yup.  There’s little reason *not* to switch trunk to JDK7 now. 
View raw message