hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11257) Deprecate 'hadoop jar'
Date Mon, 03 Nov 2014 18:34:35 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14194856#comment-14194856

Allen Wittenauer commented on HADOOP-11257:

We've had more than a few people confused that hadoop jar and yarn jar work differently due
to the fact that hadoop-env.sh and yarn-env.sh are treated as separate by the two scripts.
 This, in turn, also increases the testing matrix because one needs to verify both work as
expected. The alternative here is to eliminate yarn-env.sh, $YARN_OPTS, and related.  Needless
to say, that can't happen until trunk which means we'll be stuck with the confusion for another
handful of years.

Even though trunk currently has yarn read the hadoop-env.sh file first, YARN_OPTS takes precedence,
thus still providing some conflicting handling.  On the plus side, removing hadoop jar gives
a clear path to users as to how we expect jobs to be launched.

bq. we should not be recommending its use for MR jobs and so forth

The irony is that the mapred command (luckily?) doesn't have a way to launch jobs so mapred-env.sh
is somewhat irrelevant.  Well, unless you care that distcp, pipes, and the sampler don't use
$YARN_OPTS either...

> Deprecate 'hadoop jar'
> ----------------------
>                 Key: HADOOP-11257
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11257
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Allen Wittenauer
> Given that 'hadoop jar' and 'yarn jar' work differently, we should mark 'hadoop jar'
as deprecated in 2.7 and remove it in trunk.

This message was sent by Atlassian JIRA

View raw message