whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Periya.Data" <periya.d...@gmail.com>
Subject Re: hadoop issues on Ubuntu AMIs
Date Wed, 07 Dec 2011 05:14:13 GMT
Thanks ! A few observations:

   - After I do export conf dir and execute "hadoop fs -ls /", I see a
   different dir structure from what I see when I ssh into the machine and
   execute it as root. See outputs below.

sri@PeriyaData:~$
sri@PeriyaData:~$ export HADOOP_CONF_DIR=/\$HOME/.whirr/HadoopCluster/
sri@PeriyaData:~$
sri@PeriyaData:~$ hadoop fs -ls /
Found 25 items
-rw-------   1 root root    4767328 2011-11-02 12:55 /vmlinuz
drwxr-xr-x   - root root      12288 2011-12-03 10:49 /etc
dr-xr-xr-x   - root root          0 2011-12-02 03:28 /proc
drwxrwxrwt   - root root       4096 2011-12-05 18:07 /tmp
drwxr-xr-x   - root root       4096 2011-04-25 15:50 /srv
-rw-r--r--   1 root root   13631900 2011-11-01 22:46 /initrd.img.old
drwx------   - root root       4096 2011-11-23 22:27 /root
drwxr-xr-x   - root root       4096 2011-04-21 09:50 /mnt
drwxr-xr-x   - root root       4096 2011-12-02 09:01 /var
drwxr-xr-x   - root root       4096 2011-10-01 19:14 /cdrom
-rw-------   1 root root    4766528 2011-10-07 14:03 /vmlinuz.old
drwxr-xr-x   - root root        780 2011-12-02 16:28 /run
drwxr-xr-x   - root root       4096 2011-10-23 18:27 /usr
drwx------   - root root      16384 2011-10-01 19:05 /lost+found
drwxr-xr-x   - root root       4096 2011-11-22 22:26 /bin
drwxr-xr-x   - root root       4096 2011-04-25 15:50 /opt
drwxr-xr-x   - root root       4096 2011-10-01 19:21 /home
drwxr-xr-x   - root root       4320 2011-12-02 11:29 /dev
drwxr-xr-x   - root root       4096 2011-03-21 01:26 /selinux
drwxr-xr-x   - root root       4096 2011-11-22 22:31 /boot
drwxr-xr-x   - root root          0 2011-12-02 03:28 /sys
-rw-r--r--   1 root root   13645361 2011-11-22 22:31 /initrd.img
drwxr-xr-x   - root root       4096 2011-11-22 22:28 /lib
drwxr-xr-x   - root root       4096 2011-12-03 10:49 /media
drwxr-xr-x   - root root      12288 2011-11-22 22:29 /sbin
sri@PeriyaData:~$
sri@PeriyaData:~$

*After SSH-ing into the master node:*

sri@ip-10-90-131-240:~$ sudo su
root@ip-10-90-131-240:/home/users/sri#

root@ip-10-90-131-240::/home/users/jtv# jps
2860 Jps
2667 JobTracker
2088 NameNode
root@ip-10-90-131-240::/home/users/jtv# hadoop fs -ls /
Error: JAVA_HOME is not set.
root@ip-10-90-131-240::/home/users/jtv#

*After editing (setting java home) in the .bashrc file and sourcing it , i
get the expected dir structure:*

root@ip-10-90-131-240:/home/users/sri# hadoop fs -ls /
Found 3 items
drwxr-xr-x   - hadoop supergroup          0 2011-12-05 23:09 /hadoop
drwxrwxrwx   - hadoop supergroup          0 2011-12-05 23:08 /tmp
drwxrwxrwx   - hadoop supergroup          0 2011-12-06 01:16 /user
root@ip-10-90-131-240:/home/users/sri#
root@ip-10-90-131-240:/home/users/sri#

Is the above normal behavior?

Thanks,
PD/


*Questions:*
>>
>>    1. Assuming everything is fine, where does Hadoop gets installed on
>>    the EC2 instance? What is the path?
>>
>>
> Run jps as root and you should see the daemons running.
>
>>
>>    1. Even if Hadoop is successfully installed on the EC2 instance, are
>>    the env variables properly changed on that instance? Like, path must be
>>    updated either on its .bashrc or .bash_profile ...right?
>>
>>
> Try to run "hadoop fs -ls /" as root.
>
>

Mime
View raw message