whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From a b <autohan...@yahoo.com>
Subject Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds
Date Tue, 06 Aug 2013 17:06:34 GMT
i also took your advice and switched to the open jdk:

ab@ubuntu12-64:~$ tail ~/whirr/recipes/hadoop.properties 

#whirr.hadoop.version=1.0.4
whirr.hadoop.version=1.2.1
#whirr.hadoop.tarball.url=http://archive.apache.org/dist/hadoop/core/hadoop-${whirr.hadoop.version}/hadoop-${whirr.hadoop.version}.tar.gz
whirr.provider=aws-ec2
whirr.identity=${env:AWS_ACCESS_KEY}
whirr.credential=${env:AWS_SECRET_KEY}
whirr.location-id=us-east-1
#whirr.java.install-function=install_oracle_jdk7
whirr.java.install-function=install_openjdk

but when i launched:

 1575  whirr launch-cluster --config ~/whirr/recipes/hadoop.properties 
 1576  tail ~/whirr/recipes/hadoop.properties 
 1577  history

i got this error message on the console:

Get:36 http://archive.ubuntu.com precise-updates/multiverse Translation-en [7834 B]
Get:37 http://archive.ubuntu.com precise-updates/restricted Translation-en [2432 B]
Get:38 http://archive.ubuntu.com precise-updates/universe Translation-en [124 kB]
Fetched 7548 kB in 9s (828 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 openjdk-6-jdk : Depends: openjdk-6-jre (>= 6b27-1.12.6-1ubuntu0.12.04.2) but it is not
going to be installed
                 Recommends: libxt-dev but it is not going to be installed
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 openjdk-6-jdk : Depends: openjdk-6-jre (>= 6b27-1.12.6-1ubuntu0.12.04.2) but it is not
going to be installed
                 Recommends: libxt-dev but it is not going to be installed



i checked to make sure it was accurate:

ab@ubuntu12-64:~$  ssh -i /home/ab/.ssh/id_rsa -o "UserKnownHostsFile /dev/null" -o StrictHostKeyChecking=no
ab@54.227.132.62
Warning: Permanently added '54.227.132.62' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-48-virtual x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Tue Aug  6 16:55:22 UTC 2013

  System load:  0.06             Processes:           58
  Usage of /:   9.4% of 7.87GB   Users logged in:     0
  Memory usage: 15%              IP address for eth0: 10.164.33.99
  Swap usage:   0%

  Graph this data and manage this system at https://landscape.canonical.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

  Use Juju to deploy your cloud instances and workloads:
    https://juju.ubuntu.com/#cloud-precise

24 packages can be updated.
24 updates are security updates.

Last login: Tue Aug  6 16:53:36 2013 from 50.0.160.200
ab@ip-10-164-33-99:~$ java -version
The program 'java' can be found in the following packages:
 * default-jre
 * gcj-4.6-jre-headless
 * openjdk-6-jre-headless
 * gcj-4.5-jre-headless
 * openjdk-7-jre-headless
Ask your administrator to install one of them
ab@ip-10-164-33-99:~$ ls -l /usr/bin/j*
-rwxr-xr-x 1 root root 39440 Nov 19  2012 /usr/bin/join
-rwxr-xr-x 1 root root  3918 Mar 18 19:34 /usr/bin/json_pp
ab@ip-10-164-33-99:~$ exit

i need coaching on this - as well. i'm on a bad luck streak - i'm not buying any lotto tickets
this week.



________________________________
 From: Andrew Bayer <andrew.bayer@gmail.com>
To: user@whirr.apache.org; a b <autohandle@yahoo.com> 
Sent: Monday, August 5, 2013 5:54 PM
Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds
 


Try not building as root - that can throw things off.

A.


On Mon, Aug 5, 2013 at 5:44 PM, a b <autohandle@yahoo.com> wrote:

as for the java 7 problem - i found this suggestion:
>
>http://stackoverflow.com/questions/13225149/how-to-install-jdk-7-on-ec2-cluster-via-whirr
>
>
>i tried to download whirr - as suggested here:
>
>
>https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute
>
>
>o i did: git clone ...
>
>o i modified: core/src/main/resources/functions/...
>o i did: mvn eclipse:eclipse ...
>o i skipped: eclipse import
>o i ran: mvn install
>
>
>it fails in the "mvn install" during test
>
>
>Tests in error: 
> 
 testActionIsExecutedOnAllRelevantNodes(org.apache.whirr.actions.StopServicesActionTest):
cluster-user != root or do not run as root
>  testFilterScriptExecutionByRole(org.apache.whirr.actions.StopServicesActionTest): cluster-user
!= root or do not run as root
>  testFilterScriptExecutionByInstanceId(org.apache.whirr.actions.StopServicesActionTest):
cluster-user != root or do not run as root
>  testFilterScriptExecutionByRoleAndInstanceId(org.apache.whirr.actions.StopServicesActionTest):
cluster-user != root or do not run as root
>  testNoScriptExecutionsForNoop(org.apache.whirr.actions.StopServicesActionTest): cluster-user
!= root or do not run as root
>[..]
>  testExecuteOnlyBootstrapForNoop(org.apache.whirr.service.DryRunModuleTest): cluster-user
!= root or do not run as root
>  testExecuteOnlyBootstrapForNoopWithListener(org.apache.whirr.service.DryRunModuleTest):
cluster-user != root or do not run as root
>  testNoInitScriptsAfterConfigurationStartedAndNoConfigScriptsAfterDestroy(org.apache.whirr.service.DryRunModuleTest):
cluster-user != root or do not run as root
>  testOverrides(org.apache.whirr.command.AbstractClusterCommandTest): cluster-user !=
root or do not run as root
>
>Tests run: 99, Failures: 0, Errors: 49, Skipped: 0
>
>[INFO] ------------------------------------------------------------------------
>[INFO] Reactor Summary:
>[INFO] 
>[INFO] Apache Whirr Build Tools .......................... SUCCESS [2.612s]
>[INFO] Whirr ............................................. SUCCESS [4.488s]
>[INFO] Apache Whirr Core ................................. FAILURE [17.113s]
>[INFO] Apache Whirr Cassandra ............................ SKIPPED
>[INFO]
 Apache Whirr Ganglia .............................. SKIPPED
>[INFO] Apache Whirr Hadoop ............................... SKIPPED
>
>
>
>is there a better way to try this suggestion?
>
>
>
>
>________________________________
> From: Andrew Bayer <andrew.bayer@gmail.com>
>
>To: a b <autohandle@yahoo.com> 
>Cc: "user@whirr.apache.org" <user@whirr.apache.org> 
>Sent: Monday, August 5, 2013 4:48 PM
>
>Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds
> 
>
>
>Looks like it's the Oracle JDK7 download that's failing - http://download.oracle.com/otn-pub/java/jdk/7/jdk-7-linux-x64.tar.gz
isn't actually there. I don't know if they actually have a consistent link to get the JDK7
tarball regardless of version. I'd just use OpenJDK for now, if you can.
>
>
>A.
>
>
>On Mon, Aug 5, 2013 at 4:33 PM, a b <autohandle@yahoo.com> wrote:
>
>Get:19 http://us-east-1.ec2.archive.ubuntu.com precise/main amd64 Packages [1,273 kB]^M
>>Get:20 http://us-east-1.ec2.archive.ubuntu.com precise/universe amd64 Packages [4,786
kB]^M
>>Get:21 http://us-east-1.ec2.archive.ubuntu.com precise/main i386 Packages [1,274 kB]^M
>>Get:22 http://us-east-1.ec2.archive.ubuntu.com precise/universe i386 Packages [4,796
kB]^M
>>Get:23 http://us-east-1.ec2.archive.ubuntu.com precise/main TranslationIndex [3,706
B]^M
>>Get:24 http://us-east-1.ec2.archive.ubuntu.com precise/universe TranslationIndex [2,922
B]^M
>>Get:25 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main Sources [412 kB]^M
>>Get:26 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe Sources [93.1
kB]^M
>>Get:27 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main amd64 Packages
[672 kB]^M
>>Get:28 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe amd64 Packages
[210 kB]^M
>>Get:29 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main i386 Packages
[692 kB]^M
>>Get:30 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe i386 Packages
[214 kB]^M
>>Get:31 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main TranslationIndex
[3,564 B]^M
>>Get:32 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe TranslationIndex
[2,850 B]^M
>>Get:33 http://us-east-1.ec2.archive.ubuntu.com precise/main Translation-en [726 kB]^M
>>Get:34 http://us-east-1.ec2.archive.ubuntu.com precise/universe Translation-en [3,341
kB]^M
>>Get:35 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main Translation-en
[298 kB]^M
>>Get:36 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe Translation-en
[123 kB]^M
>>Fetched 26.1 MB in 21s (1,241 kB/s)^M
>>Reading package lists...^M
>>Could not download  http://apache.osuosl.org/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz.md5.
Continuing.^M
>>, error=^M
>>gzip: stdin: not in gzip format^M
>>tar: Child returned status 1^M
>>tar: Error is not recoverable: exiting now^M
>>mv: cannot stat `jdk1*': No such file or directory^M
>>update-alternatives: error: alternative path /usr/lib/jvm/java-7-oracle/bin/java doesn't
exist.^M
>>
>>
>>
>>
>>________________________________
>> From: Andrew Bayer <andrew.bayer@gmail.com>
>>To: user@whirr.apache.org; a b <autohandle@yahoo.com> 
>>Sent: Monday, August 5, 2013 4:22 PM
>>
>>Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds
>> 
>>
>>
>>Ok, Whirr does try to download the md5 file, but if it fails to find it, that's not
a blocking error - it'll keep going anyway. What's after that in the logs?
>>
>>
>>A.
>>
>>
>>On Mon, Aug 5, 2013 at 4:19 PM, a b <autohandle@yahoo.com> wrote:
>>
>>ok - i'm not sure what you are asking.
>>>
>>>whirr launch-cluster --config
~/whirr/recipes/hadoop.properties 
>>>
>>>
>>>
>>>where these are the properties i think i changed or added from the original recipe:
>>>
>>>
>>>whirr.cluster-name=hadoop-ec2
>>>whirr.instance-templates=1
hadoop-namenode+hadoop-jobtracker,1 hadoop-datanode+hadoop-tasktracker
>>>whirr.hardware-id=t1.micro
>>>whirr.image-id=us-east-1/ami-25d9a94c
>>>whirr.hadoop.version=1.2.1
>>>whirr.provider=aws-ec2
>>>whirr.identity=${env:AWS_ACCESS_KEY} 
>>>whirr.credential=${env:AWS_SECRET_KEY}
>>>whirr.location-id=us-east-1
>>>whirr.java.install-function=install_oracle_jdk7
>>>
>>>
>>>
>>>________________________________
>>> From: Andrew Bayer <andrew.bayer@gmail.com>
>>>To: user@whirr.apache.org; a b <autohandle@yahoo.com> 
>>>Sent: Monday, August 5, 2013 4:11 PM
>>>Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds
>>> 
>>>
>>>
>>>Ok, it looks like they've actually been doing .mds for a while. Where are you
seeing this error?
>>>
>>>
>>>A.
>>>
>>>
>>>On Mon, Aug 5, 2013 at 4:09 PM, Andrew Bayer <andrew.bayer@gmail.com> wrote:
>>>
>>>That looks like they broke the hadoop-1.2.1 release - the file should be .md5.
I'd bug the Hadoop project about that.
>>>>
>>>>A.
>>>>
>>>>
>>>>
>>>>On Mon, Aug 5, 2013 at 3:42 PM, a b <autohandle@yahoo.com> wrote:
>>>>
>>>>i get a whirr error:
>>>>>
>>>>>
>>>>>Could not download  http://apache.osuosl.org/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz.md5
>>>>>
>>>>>
>>>>>when i browse over to: http://apache.osuosl.org/hadoop/common/hadoop-1.2.1/
>>>>>i can see the file is named: hadoop-1.2.1.tar.gz.mds
>>>>>
>>>>>
>>>>>how do i tell whirr to use a different suffix?
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
Mime
View raw message