hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: should clients be able to override *-site.xml?
Date Wed, 10 Sep 2014 23:34:45 GMT
I participated on the HADOOP-9450 code review.  There was some debate about
the significance of keeping HADOOP_CONF_DIR first.  Ultimately, there was
agreement that "FIRST" really ought to mean first.  Apparently, other
Hadoop ecosystem projects have made the same interpretation and implemented
their scripts accordingly.  Just echoing what Allen said, there were always
plenty of ways for a user to override *-site.xml too.

Of course, I'm sorry to hear the change caused an issue at your company,

Chris Nauroth

On Wed, Sep 10, 2014 at 10:55 AM, Allen Wittenauer <aw@altiscale.com> wrote:

> On Sep 10, 2014, at 10:14 AM, Sangjin Lee <sjlee@apache.org> wrote:
> > set and the user provides his/her version of *-site.xml through
> > HADOOP_CLASSPATH, the user would end up trumping the hadoop
> configuration.
> > And I believe this behavior is preserved after Allen's changes
> > (HADOOP-9902).
> >
>         I'd be surprised if 9902 changed this behavior, especially given
> that HADOOP_CONF_DIR is added fairly quickly to the CLASSPATH and
> HADOOP_USER_CLASSPATH is processed last or near last.
> > Is this an intended behavior? What I'm not sure of is whether we expect
> the
> > client to be able to override the site.xml files the hadoop configuration
> > provides. If that is true, then it is working as desired. If not, we'd
> need
> > to fix this behavior.
> >
> > Thoughts?
> >
>         Users have always been able to override the *-site.xml files via
> java properties, jobconf/etc,  pointing HADOOP_CONF_DIR to somewhere else,
> or providing --conf flags to shell commands.  In other words, this isn't
> new behavior, but provides yet another option to override the provided
> settings.

NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message