hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Looking to a Hadoop 3 release
Date Fri, 19 Feb 2016 14:19:13 GMT

> On 19 Feb 2016, at 11:27, Dmitry Sivachenko <trtrmitya@gmail.com> wrote:
>> On 19 Feb 2016, at 01:35, Andrew Wang <andrew.wang@cloudera.com> wrote:
>> Hi all,
>> Reviving this thread. I've seen renewed interest in a trunk release since
>> HDFS erasure coding has not yet made it to branch-2. Along with JDK8, the
>> shell script rewrite, and many other improvements, I think it's time to
>> revisit Hadoop 3.0 release plans.

It's time to start ... I suspect it'll take a while to stabilise. I look forward to the new
shell scripts already

One thing I do want there is for all the alpha releases to make clear that there are no compatibility
policies here; protocols may change and there is no requirement of the first 3.x release to
be compatible with all the 3.0.x alphas. That's something we missed out on the 2.0.x-alpha
process, or at least not repeated often enough.

> Hello,
> any chance IPv6 support (HADOOP-11890) will be finished before 3.0 comes out?
> Thanks!

sounds like a good time for a status update on the FB work —and anything people can do to
test it would be appreciated by all. That includes testing on ipv4 systems, and especially,
IPv4/v6 systems with Kerberos turned on and both MIT and AD kerberos servers. At the same
time, IPv6 support ought to be something that could be added in.

I don't have any opinions on timescale, but

+1 to anything related to classpath isolation
+1 to a careful bump of versions of dependencies.
+1 to fixing the outstanding Java 8 migration issues, especially the big Jersey patch that's
just been updated.
+1 to switching to JIRA-created release notes

Having been doing the slider releases recently, it's clear to me that you can do a lot in
automating the release process itself. All those steps in the release runbook can be turned
into targets in a special ant release.xml build file, calling maven, gpg, etc.

I think doing something like this for 3.0 will significantly benefit both the release phase
here but the future releases

This is the slider one: https://github.com/apache/incubator-slider/blob/develop/bin/release.xml

It doesn't replace maven, instead it choreographs that along with all the other steps: signing
and checksumming artifacts, publishing them, voting

it includes
 -refusing to release if the git repo is modified
 -making the various git branch/tag/push operations
 -issuing the various mvn versions:update commands
 -publishing via asf SVN 
 -using GET calls too verify the artifacts made it
 -generating the vote and vote result emails (it even counts the votes)

I recommend this is included as part of the release process. It does make a difference; we
can now cut new releases with no human intervention other than editing a properties file and
running different targets as the process goes through its release and vote phases.

View raw message