whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Savu <savu.and...@gmail.com>
Subject Re: Java and Hadoop on Ubuntu 11.10 AMI
Date Sun, 04 Dec 2011 20:44:05 GMT
Here are a few advices:

* use Ubuntu 10.04 LTS 64bit on the cluster nodes - this is the OS version
we are testing against before making a release
* whirr.hadoop.tarball.url is irrelevant if you specify
whirr.hadoop.install-function. Same is true for whirr.hadoop.version.
* install_cdh_hadoop will install by default the latest release.
We've implemented a way of selecting the version in 0.7.0:
https://issues.apache.org/jira/browse/WHIRR-415
* use something bigger than t1.micro - this is just too small to run Hadoop

Let me know if you are still having problems.

-- Andrei Savu / andreisavu.ro

On Sun, Dec 4, 2011 at 9:44 PM, Periya.Data <periya.data@gmail.com> wrote:

> Hi,
>    I am trying Whirr to spin-up EC2 clusters. Here are the specs:
>
>
>    - Local laptop - Ubuntu 11.10 Oneiric Ocelot + Hadoop-0.20.2-cdh3u2
>    - EC2 Whirr config :
>    -
>       - whirr.hadoop.install-function=install_cdh_hadoop
>    - whirr.hadoop.configure-function=configure_cdh_hadoop
>       - whirr.hadoop.version=0.20.2
>       - whirr.hadoop.tarball.url=
>       http://archive.cloudera.com/cdh/3/hadoop-0.20.2-cdh3u2.tar.gz
>       - whirr.hardware-id=t1.micro
>                 # Ubuntu 11.10 Oneiric EBS boot from alestic
>    - whirr.image-id=us-east-1/ami-a7f539ce
>       - whirr.location-id=us-east-1
>
>
> *Problem:* The cluster seems to be up and running. But, cannot see Java
> and Hadoop on the instance.
>
> *Question:* Is this a problem or should I clearly mention in my Whirr
> config/properties file what version of JDK to be installed and its download
> URL?
>
>
> *Some logs:* (I ssh-ed into one of the nodes and tried to see the logs)
>
> sri@domU-12-31-39-02-21-20:/tmp$ ls -al
> total 40
> drwxrwxrwt  4 root root  4096 2011-12-04 06:25 .
> drwxr-xr-x 22 root root  4096 2011-12-04 05:46 ..
> -rw-r--r--  1 root root   938 2011-12-04 05:46 core-site.xml
> -rw-r--r--  1 root root   701 2011-12-04 05:46 hdfs-site.xml
> drwxr-xr-x  2 root root  4096 2011-12-04 05:46 jclouds-script-1322977554131
> drwxr-xr-x  2 root root  4096 2011-12-04 05:44 logs
> -rw-r--r--  1 root root  1447 2011-12-04 05:46 mapred-site.xml
> -rwxr--r--  1 root root 11212 2011-12-04 05:44 setup-sri.sh
> sri@domU-12-31-39-02-21-20:/tmp$
> sri@domU-12-31-39-02-21-20:/tmp$
> sri@domU-12-31-39-02-21-20:/tmp$
> sri@domU-12-31-39-02-21-20:/tmp$ cd logs/
> sri@domU-12-31-39-02-21-20:/tmp/logs$ ls -l
> total 12
> -rw-r--r-- 1 root root  117 2011-12-04 05:45 stderr.log
> -rw-r--r-- 1 root root 4535 2011-12-04 05:45 stdout.log
> sri@domU-12-31-39-02-21-20:/tmp/logs$ more stderr.log
> E: Package 'sun-java6-jdk' has no installation candidate
> /tmp/setup-sri.sh: line 161: java: command not found
> sri@domU-12-31-39-02-21-20:/tmp/logs$
>
> ----------
>
> Get:36 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe
> i386 Packages [57.2 kB]
> Get:37 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/multiverse
> i386 Packages [4396 B]
> Get:38 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/main
> TranslationIndex [73 B]
> Get:39 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/multiverse
> TranslationIndex [72 B
> ]
> Get:40 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe
> TranslationIndex [73 B]
> Get:41 http://us-east-1.ec2.archive.ubuntu.com oneiric/main
> Translation-en [701 kB]
> Hit http://us-east-1.ec2.archive.ubuntu.com oneiric/multiverse
> Translation-en
> Get:42 http://us-east-1.ec2.archive.ubuntu.com oneiric/universe
> Translation-en [3165 kB]
> Get:43 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/main
> Translation-en [99.2 kB]
> Get:44 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/multiverse
> Translation-en [2419 B
> ]
> Get:45 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe
> Translation-en [35.4 kB]
> Fetched 16.2 MB in 12s (1290 kB/s)
> Reading package lists...
> Reading package lists...
> Building dependency tree...
> Reading state information...
> Package sun-java6-jdk is not available, but is referred to by another
> package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
>
> sri@domU-12-31-39-02-21-20:/tmp/logs$
> Get:36 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe
> i386 Packages [57.2 kB]
> Get:37 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/multiverse
> i386 Packages [4396 B]
> Get:38 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/main
> TranslationIndex [73 B]
> Get:39 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/multiverse
> TranslationIndex [72 B]
> Get:40 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe
> TranslationIndex [73 B]
> Get:41 http://us-east-1.ec2.archive.ubuntu.com oneiric/main
> Translation-en [701 kB]
> Hit http://us-east-1.ec2.archive.ubuntu.com oneiric/multiverse
> Translation-en
> Get:42 http://us-east-1.ec2.archive.ubuntu.com oneiric/universe
> Translation-en [3165 kB]
> Get:43 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/main
> Translation-en [99.2 kB]
> Get:44 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/multiverse
> Translation-en [2419 B]
> Get:45 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe
> Translation-en [35.4 kB]
> Fetched 16.2 MB in 12s (1290 kB/s)
> Reading package lists...
> Reading package lists...
> Building dependency tree...
> Reading state information...
> Package sun-java6-jdk is not available, but is referred to by another
> package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
>
> sri@domU-12-31-39-02-21-20:/tmp/logs$
>
> *[On a different cluster spun-up at a later time]*
>
> sri@domU-12-31-39-10-6D-38:~$ java
> The program 'java' can be found in the following packages:
>  * gcj-4.4-jre-headless
>  * gcj-4.6-jre-headless
>  * openjdk-6-jre-headless
>  * gcj-4.5-jre-headless
>  * openjdk-7-jre-headless
> Ask your administrator to install one of them
> sri@domU-12-31-39-10-6D-38:~$ java -version
> The program 'java' can be found in the following packages:
>  * gcj-4.4-jre-headless
>  * gcj-4.6-jre-headless
>  * openjdk-6-jre-headless
>  * gcj-4.5-jre-headless
>  * openjdk-7-jre-headless
> Ask your administrator to install one of them
> sri@domU-12-31-39-10-6D-38:~$
>
> sri@domU-12-31-39-10-6D-38:~$ hadoop version
> hadoop: command not found
> sri@domU-12-31-39-10-6D-38:~$
>
> ===========================================
>
> Finally, a quick Googling pointed to a JIRA issue: https://issues.apache.org/jira/browse/WHIRR-435
>
>
> I am seeing the same stuff. Any updates? If this is a real issues, I do
> not have any problems in using Ubuntu 11.04 AMI for now.
>
>
> Your help is very much appreciated.
>
>
> Thanks,
>
> PD.
>

Mime
View raw message