From common-issues-return-66825-apmail-hadoop-common-issues-archive=hadoop.apache.org@hadoop.apache.org Fri Aug 8 03:59:16 2014 Return-Path: X-Original-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 378DD113A5 for ; Fri, 8 Aug 2014 03:59:16 +0000 (UTC) Received: (qmail 21155 invoked by uid 500); 8 Aug 2014 03:59:15 -0000 Delivered-To: apmail-hadoop-common-issues-archive@hadoop.apache.org Received: (qmail 21106 invoked by uid 500); 8 Aug 2014 03:59:15 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-issues@hadoop.apache.org Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 21093 invoked by uid 99); 8 Aug 2014 03:59:15 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 08 Aug 2014 03:59:15 +0000 Date: Fri, 8 Aug 2014 03:59:15 +0000 (UTC) From: "Allen Wittenauer (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-9902) Shell script rewrite MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14090273#comment-14090273 ] Allen Wittenauer commented on HADOOP-9902: ------------------------------------------ bq. It seems that hadoop-functions.sh ends up in sbin/hadoop-functions.sh in the finally binary assembly but hadoop-config.sh looks for it in the HADOOP_LIBEXEC_DIR. Weird! I've been using tar ball builds and it shows up in HADOOP_LIBEXEC_DIR / libexec as expected. I wonder what's going on here then. hadoop-dist.xml even specifies libexec and excludes it from sbin specifically. From my own fresh build of trunk + this patch: {code} mbp:hadoop-3.0.0-SNAPSHOT aw$ pwd /Users/aw/Src/hadoop-common/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT mbp:hadoop-3.0.0-SNAPSHOT aw$ find . -name hadoop-functions.sh ./libexec/hadoop-functions.sh {code} So I guess I need some advice here on a way to reproduce it and/or how to fix this one. bq. hadoop-common-project/hadoop-common/src/main/bin/hadoop's extra quotes Yup, that's clearly broken. Probably snuck in while I was playing with the space bug. That's an easy fix. bq. HADOOPOSTYPE needs to be documents This is one of those things that I didn't know what to do with it. I really don't want it to exist, to be honest, but with HADOOP-8719, we have to do something in the hadoop-env.sh. Stupid Apple and/or Oracle. I think in the end we're just kind of screwed and will need to make it real. :/ I'll change it to HADOOP_OS_TYPE, define it both in hadoop-env.sh and hadoop-functions.sh so that it can be referenced as a real value. bq. Are we planning to use *-env.sh for documenting all the variables that one may set? That was my intent, yes. I'm looking at it from the perspective of your typical /etc/default/*, /etc/sysconfig, etc, type file. bq. Any reason populate_slaves_file function is in hadoop-config.sh and not in hadoop-functions.sh ? It was really meant as a utility function only for hadoop-config.sh to simplify the options listing/parsing code. But there's no reason it can't be moved. I'll do that as well as make it hadoop_ to keep the name space clean. bq. The following appears to be a sort of no-op if value is not set Yup, correct. Another easy fix. (... and clearly bad copypasta from the current code ...) bq. Any reason not to try harder and see what type -p java returns? I think it's too risky. I'd much rather have an explicit JAVA_HOME than assume that /usr/bin/java (or whatever happens to get returned) is "good". 'type -p' can also be wildly unpredictable since it uses the hashed value... this is not necessarily the first one that shows up in the path! Thanks! > Shell script rewrite > -------------------- > > Key: HADOOP-9902 > URL: https://issues.apache.org/jira/browse/HADOOP-9902 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts > Affects Versions: 3.0.0 > Reporter: Allen Wittenauer > Assignee: Allen Wittenauer > Labels: releasenotes > Attachments: HADOOP-9902-10.patch, HADOOP-9902-11.patch, HADOOP-9902-12.patch, HADOOP-9902-13-branch-2.patch, HADOOP-9902-13.patch, HADOOP-9902-2.patch, HADOOP-9902-3.patch, HADOOP-9902-4.patch, HADOOP-9902-5.patch, HADOOP-9902-6.patch, HADOOP-9902-7.patch, HADOOP-9902-8.patch, HADOOP-9902-9.patch, HADOOP-9902.patch, HADOOP-9902.txt, hadoop-9902-1.patch, more-info.txt > > > Umbrella JIRA for shell script rewrite. See more-info.txt for more details. -- This message was sent by Atlassian JIRA (v6.2#6252)