hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Kambatla <ka...@cloudera.com>
Subject Re: [DISCUSS] Clarification on Compatibility Policy: Upgraded Client + Old Server
Date Wed, 19 Mar 2014 21:26:23 GMT
+1 on supporting new clients with old servers of the same major version,
and updating the policy to capture that clearly.


On Wed, Mar 19, 2014 at 1:59 PM, Chris Nauroth <cnauroth@hortonworks.com>wrote:

> I'd like to discuss clarification of part of our compatibility policy.
>  Here is a link to the compatibility documentation for release 2.3.0:
>
>
> http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-common/Compatibility.html#Wire_compatibility
>
> For convenience, here are the specific lines in question:
>
> Client-Server compatibility is required to allow users to continue using
> the old clients even after upgrading the server (cluster) to a later
> version (or vice versa). For example, a Hadoop 2.1.0 client talking to a
> Hadoop 2.3.0 cluster.
>
> Client-Server compatibility is also required to allow upgrading individual
> components without upgrading others. For example, upgrade HDFS from version
> 2.1.0 to 2.2.0 without upgrading MapReduce.
>
> Server-Server compatibility is required to allow mixed versions within an
> active cluster so the cluster may be upgraded without downtime in a rolling
> fashion.
>
> Notice that there is no specific mention of upgrading the client ahead of
> the server.  (There is no clause for "upgraded client + old server".)
>  Based on my experience, this is a valid use case when a user wants to pick
> up a client-side bug fix ahead of the cluster administrator's upgrade
> schedule.
>
> Is it our policy to maintain client compatibility with old clusters within
> the same major release?  I think many of us have assumed that the answer is
> yes and coded our new features accordingly, but it isn't made explicit in
> the documentation.  Do we all agree that the answer is yes, or is it
> possibly up for debate depending on the change in question?  In RFC 2119
> lingo, is it a MUST or a SHOULD?  Either way, I'd like to update the policy
> text to make our decision clear.  After we have consensus, I can volunteer
> to file an issue and patch the text of the policy.
>
> This discussion started initially in MAPREDUCE-4052, which involved
> changing our scripting syntax for MapReduce YARN container submissions.  We
> settled the question there by gating the syntax change behind a
> configuration option.  By default, it will continue using the existing
> syntax currently understood by the pre-2.4.0 NodeManager, thus preserving
> compatibility.  We wanted to open the policy question for wider discussion
> though.
>
> Thanks, everyone.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message