tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Lowe <jl...@oath.com.INVALID>
Subject Re: [DISCUSS] Aligning master with Hadoop 3 and create separate 0.9.x line
Date Mon, 26 Mar 2018 15:09:21 GMT
+1 for incrementing the required Hadoop version from 2.7 as long as we
continue to push bugfixes to the 0.9 line for a while.  We currently
have a "hadoop28" profile in Tez which is mostly compatible with
Hadoop 3.x, but it does not get much testing.  There is no release
vehicle for it, and it does not even get tested from the precommit
build.  Promoting this or a 3.x profile to the main build is the most
straightforward way to get it tested and released in an
easy-to-consume form.

This does mean we would need to maintain two release lines for a
while, at least until users and downstream projects migrate away from
Hadoop 2.7.  We've done two lines before (even three, if we consider
the days of 0.9.x, 0.8.x, and 0.7.x all co-existing), and in this case
I think the cost of maintaining those two lines is worth it to move
the project forward as the stack migrates to Hadoop 3.x.

Jason



On Thu, Mar 22, 2018 at 7:16 PM, Eric Wohlstadter
<ewohlstadter@hortonworks.com> wrote:
> Hi all,
> I’d like to propose that we move towards aligning the Tez master branch with support
for Hadoop 3+ only.
>  A separate branch and distribution (e.g. on Maven Central) would be created to maintain
the 0.9.x line with support for Hadoop 2.7+.
>
> This will help ensure that Tez can continue to move forward with other progress in the
greater Hadoop community.
> Since Hadoop 3 is not backward compatible with Hadoop 2, my opinion is that it is too
difficult for Tez to maintain such backward compatibility
>
>
>   *   Tez master branch would support only Hadoop 3+ moving forward
>   *   Bug fixes would be required to be pushed to both to master and the 0.9.x line
>   *   Major feature or performance improvements would be required to be pushed to both
master and the 0.9.x line (unless they require Hadoop 3+)
>   *   Minor feature or performance improvements can be pushed only to master
>   *   A new release with Hadoop 3+ only support would be placed on high priority (possibly
0.10?)
>      *   At a minimum the issues under TEZ-3903 would be required
>
> Please help to provide any feedback or comments about this unofficial proposal.
> This is not an official vote but it would help to get people’s thoughts/questions or
unofficial (+1, -1).
>

Mime
View raw message