hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
Date Wed, 07 Sep 2011 01:54:10 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098537#comment-13098537

Devaraj Das commented on HADOOP-7599:

Some comments:
1. The mapred system dir should be 700ed 
2. The -format is always called by default in hadoop-setup-hdfs. Can we make the formatting
done based on command line option (-format) provided by the user running the script. I really
don't want implicit namenode formatting.
3. We could let the directory /mapred be group owned by the system group.
4. The --datanodes & --tasktrackers options - could they be made optional? They might
be already optional. Please confirm.
5. Could we live with namenode-url instead of replacing it with namenode-host (ditto for jobtracker-url).
I also see you changed a couple of other places in a backward incompatible way (like HADOOP_JT_HOST,
taskcontroller ownership). Want to avoid incompatible changes.
6. Why did we remove the call to hadoop-setup-config.sh from src/packages/hadoop-setup-conf.sh
7. The group is hardcoded to 'hadoop' in a couple of places. Can we avoid that?
8. I see commented-out lines in hadoop-setup-conf.sh. Please remove those.
9. I don't think we should blindly overwrite the conf files in the config directory. We probably
should warn & exit if the conf directory already has some files within. The user can probably
use --force if he wants to avoid the warning.
10. All the components in the path leading up to taskcontroller.cfg has to be owned by root.
Have this checked.
11. Where is HADOOP_SECURE_DN_LOG_DIR used?
12. A whole lot of configuration options that you added in the *-site.xml files are already
there in *-default.xml. We don't need those. We only need the security related ones. Also,
we don't want to play with the non-security configs like mapred.tasktracker.map.tasks.maximum.

> Improve hadoop setup conf script to setup secure Hadoop cluster
> ---------------------------------------------------------------
>                 Key: HADOOP-7599
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7599
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>    Affects Versions:
>         Environment: Java 6, RHEL 5.6
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>             Fix For:
>         Attachments: HADOOP-7599.patch
> Setting up a secure Hadoop cluster requires a lot of manual setup.  The motivation of
this jira is to provide setup scripts to automate setup secure Hadoop cluster.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message