hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Shvachko <shv.had...@gmail.com>
Subject Re: Heads up - 2.0.5-beta
Date Wed, 01 May 2013 23:08:30 GMT
On Wed, May 1, 2013 at 1:15 PM, Arun C Murthy <acm@hortonworks.com> wrote:
> On Apr 30, 2013, at 4:28 PM, Konstantin Shvachko wrote:
> > If the next release has to be 2.0.5 I would like to make an alternative
> > proposal, which would include
> > - stabilization of current 2.0.4
> > - making all API changes to allow freezing them post 2.0.5
> > And nothing else.
> I think it's hard to clearly define - 'nothing else'. For e.g.
YARN-398/YARN-392. It's a 'feature' but worth putting in right-away since
it so low-risk. MAPREDUCE-5108 is a 'feature' but is critical for ensuring
a smooth transition from MR1 to MR2 etc. etc.

Don't see contradictions to the plan here.
Both YARN-398, YARN-392 are important optimizations. They require API
changes, so it is better to commit them into 2.0.5. If RM sees a low risk
in including the implementations, I don't see a problem.
MAPREDUCE-5108 as a compatibility issue should go in, imho.

> Rather than get tied up in knots, it would be useful to go with API
changes as *mandatory* and everything as optional and not hold up the
release for them (which is what we have done in hadoop-2.x since forever).
IAC, risk should be quantified by people working on individual jiras.

People were and are complaining that every release 2.0 was incompatible
with the previous.
I would not say any API changes, but those that help locking them post 2.0.5
"everything as optional" is too wide in my understanding as it can be
anything, including changes that break downstream components. In order to
avoid that, the changes should be minimized to bug fixes.

> Also, it will be useful to actually start testing things as they stand
rather than continue to discuss endlessly - would you be willing to help
test on of hadoop-2.x? If so, could you please share your plans? I'm sure
everyone will appreciate it.

Thank you for asking.
We did comprehensive testing internally of hadoop 2.0.3 and hadoop 2.0.4 as
they stand now using standard Hadoop tools and BigTop for integration. Cos
introduced Jenkins build for branch 2, which wasn't set up.
Testing things as they currently EVOLVE doesn't make sense to me, as the
volume of changes proposed will invalidate any current testing.
Endless discussions are not productive. I put up the vote for this release


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message