whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Savu <savu.and...@gmail.com>
Subject Re: Whirr 0.7.1 : CDH : Rackspace : Problem
Date Thu, 21 Jun 2012 11:56:39 GMT
Try replacing:

whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1
hadoop-datanode+hadoop-tasktracker

with:

whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,1
hadoop-datanode+hadoop-tasktracker

The role order is important.

-- Andrei Savu / axemblr.com / Tools for Clouds


On Thu, Jun 21, 2012 at 2:52 PM, Paritosh Ranjan <pranjan@xebia.com> wrote:

>  Sometimes the namenode is up but the jobtracker has never been up.
> I have tried it 4-5 times.
>  ------------------------------
> *From:* Andrei Savu [savu.andrei@gmail.com]
> *Sent:* Thursday, June 21, 2012 1:49 PM
> *To:* user@whirr.apache.org
> *Subject:* Re: Whirr 0.7.1 : CDH : Rackspace : Problem
>
>  Have you checked the servers? I see no serious error. The file
> permissions issue should be harmless.
>
>  By default Whirr is deploying CDH3U4. Is this what you need?
>
> -- Andrei Savu / axemblr.com / Tools for Clouds
>
> On Thu, Jun 21, 2012 at 2:43 PM, Paritosh Ranjan <pranjan@xebia.com>wrote:
>
>>  This is the complete log, if it can help.
>>
>>  ./whirr launch-cluster --config hadoop.properties
>> Bootstrapping cluster
>> Configuring template
>> Configuring template
>> Starting 1 node(s) with roles [hadoop-datanode, hadoop-tasktracker]
>> Starting 1 node(s) with roles [hadoop-jobtracker, hadoop-namenode]
>> Nodes started: [[id=20933521, providerId=20933521,
>> group=humaninferencerackspace, name=humaninferencerackspace-af5,
>> location=[id=e90923cb40144ee405a900686d6cd631, scope=HOST,
>> description=e90923cb40144ee405a900686d6cd631, parent=cloudservers-us,
>> iso3166Codes=[], metadata={}], uri=null, imageId=112, os=[name=null,
>> family=ubuntu, version=10.04, arch=null, is64Bit=true, description=Ubuntu
>> 10.04 LTS], state=RUNNING, loginPort=22,
>> hostname=humaninferencerackspace-af5, privateAddresses=[10.183.34.9],
>> publicAddresses=[50.57.166.152], hardware=[id=4, providerId=4, name=2GB
>> server, processors=[[cores=8.0, speed=1.0]], ram=2048, volumes=[[id=null,
>> type=LOCAL, size=80.0, device=null, durable=true, isBootDevice=true]],
>> supportsImage=ALWAYS_TRUE, tags=[]], loginUser=root, userMetadata={},
>> tags=[]]]
>> Nodes started: [[id=20933520, providerId=20933520,
>> group=humaninferencerackspace, name=humaninferencerackspace-d4b,
>> location=[id=c948bbc4457a5a9a1d135260d0f5ec01, scope=HOST,
>> description=c948bbc4457a5a9a1d135260d0f5ec01, parent=cloudservers-us,
>> iso3166Codes=[], metadata={}], uri=null, imageId=112, os=[name=null,
>> family=ubuntu, version=10.04, arch=null, is64Bit=true, description=Ubuntu
>> 10.04 LTS], state=RUNNING, loginPort=22,
>> hostname=humaninferencerackspace-d4b, privateAddresses=[10.183.34.3],
>> publicAddresses=[50.57.166.89], hardware=[id=4, providerId=4, name=2GB
>> server, processors=[[cores=8.0, speed=1.0]], ram=2048, volumes=[[id=null,
>> type=LOCAL, size=80.0, device=null, durable=true, isBootDevice=true]],
>> supportsImage=ALWAYS_TRUE, tags=[]], loginUser=root, userMetadata={},
>> tags=[]]]
>> Authorizing firewall ingress to [20933520] on ports [50030] for [
>> 108.171.174.174/32]
>> Authorizing firewall ingress to [20933520] on ports [8021] for [
>> 50.57.166.89/32]
>> Authorizing firewall ingress to [20933520] on ports [50070] for [
>> 108.171.174.174/32]
>> Authorizing firewall ingress to [20933520] on ports [8020, 8021] for [
>> 50.57.166.89/32]
>> Authorizing firewall ingress to [20933520] on ports [50030] for [
>> 108.171.174.174/32]
>> Authorizing firewall ingress to [20933520] on ports [8021] for [
>> 50.57.166.89/32]
>> Starting to run scripts on cluster for phase configureinstances: 20933521
>> Starting to run scripts on cluster for phase configureinstances: 20933520
>> Running configure phase script on: 20933521
>> Running configure phase script on: 20933520
>> configure phase script run completed on: 20933521
>> configure phase script run completed on: 20933520
>>
>> Successfully executed configure script: [output=        at
>> java.io.FileOutputStream.openAppend(Native Method)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:207)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
>>         at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
>>         at
>> org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
>>         at
>> org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:216)
>>         at
>> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
>>         at
>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
>> hadoop-0.20-namenode.
>> Safe mode is OFF
>> , error=12/06/21 10:58:38 INFO namenode.FSNamesystem:
>> supergroup=supergroup
>> 12/06/21 10:58:38 INFO namenode.FSNamesystem: isPermissionEnabled=true
>> 12/06/21 10:58:38 INFO namenode.FSNamesystem:
>> dfs.block.invalidate.limit=1000
>> 12/06/21 10:58:38 INFO namenode.FSNamesystem: isAccessTokenEnabled=false
>> accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
>> 12/06/21 10:58:38 INFO common.Storage: Image file of size 110 saved in 0
>> seconds.
>> 12/06/21 10:58:38 INFO common.Storage: Storage directory
>> /data/hadoop/hdfs/name has been successfully formatted.
>> 12/06/21 10:58:38 INFO namenode.NameNode: SHUTDOWN_MSG:
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down NameNode at
>> 50-57-166-89.static.cloud-ips.com/50.57.166.89
>> ************************************************************/
>> , exitCode=0]
>> Successfully executed configure script:
>> [output=java.io.FileNotFoundException:
>> /var/log/hadoop/logs/SecurityAuth.audit (Permission denied)
>>         at java.io.FileOutputStream.openAppend(Native Method)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:207)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
>>         at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
>>         at
>> org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
>>         at
>> org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:216)
>>         at
>> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
>>         at
>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
>> hadoop-0.20-tasktracker.
>>  , error=dpkg-preconfigure: unable to re-open stdin:
>> update-rc.d: warning: hadoop-0.20-datanode start runlevel arguments (2 3
>> 4 5) do not match LSB Default-Start values (3 5)
>> update-rc.d: warning: hadoop-0.20-datanode stop runlevel arguments (0 1
>> 6) do not match LSB Default-Stop values (0 1 2 4 6)
>> dpkg-preconfigure: unable to re-open stdin:
>> update-rc.d: warning: hadoop-0.20-tasktracker start runlevel arguments (2
>> 3 4 5) do not match LSB Default-Start values (3 5)
>> update-rc.d: warning: hadoop-0.20-tasktracker stop runlevel arguments (0
>> 1 6) do not match LSB Default-Stop values (0 1 2 4 6)
>> , exitCode=0]
>> Finished running configure phase scripts on all cluster instances
>> Completed configuration of humaninference_rackspace role hadoop-jobtracker
>> Jobtracker web UI available at
>> http://50-57-166-89.static.cloud-ips.com:50030
>> Completed configuration of humaninference_rackspace role hadoop-namenode
>> Namenode web UI available at
>> http://50-57-166-89.static.cloud-ips.com:50070
>> Wrote Hadoop site file
>> /home/hadoepje/.whirr/humaninference_rackspace/hadoop-site.xml
>> Wrote Hadoop proxy script
>> /home/hadoepje/.whirr/humaninference_rackspace/hadoop-proxy.sh
>> Completed configuration of humaninference_rackspace role hadoop-datanode
>> Completed configuration of humaninference_rackspace role
>> hadoop-tasktracker
>> Wrote instances file
>> /home/hadoepje/.whirr/humaninference_rackspace/instances
>> Started cluster of 2 instances
>> Cluster{instances=[Instance{roles=[hadoop-datanode, hadoop-tasktracker],
>> publicIp=50.57.166.152, privateIp=10.183.34.9, id=20933521,
>> nodeMetadata=[id=20933521, providerId=20933521,
>> group=humaninferencerackspace, name=humaninferencerackspace-af5,
>> location=[id=e90923cb40144ee405a900686d6cd631, scope=HOST,
>> description=e90923cb40144ee405a900686d6cd631, parent=cloudservers-us,
>> iso3166Codes=[], metadata={}], uri=null, imageId=112, os=[name=null,
>> family=ubuntu, version=10.04, arch=null, is64Bit=true, description=Ubuntu
>> 10.04 LTS], state=RUNNING, loginPort=22,
>> hostname=humaninferencerackspace-af5, privateAddresses=[10.183.34.9],
>> publicAddresses=[50.57.166.152], hardware=[id=4, providerId=4, name=2GB
>> server, processors=[[cores=8.0, speed=1.0]], ram=2048, volumes=[[id=null,
>> type=LOCAL, size=80.0, device=null, durable=true, isBootDevice=true]],
>> supportsImage=ALWAYS_TRUE, tags=[]], loginUser=root, userMetadata={},
>> tags=[]]}, Instance{roles=[hadoop-jobtracker, hadoop-namenode],
>> publicIp=50.57.166.89, privateIp=10.183.34.3, id=20933520,
>> nodeMetadata=[id=20933520, providerId=20933520,
>> group=humaninferencerackspace, name=humaninferencerackspace-d4b,
>> location=[id=c948bbc4457a5a9a1d135260d0f5ec01, scope=HOST,
>> description=c948bbc4457a5a9a1d135260d0f5ec01, parent=cloudservers-us,
>> iso3166Codes=[], metadata={}], uri=null, imageId=112, os=[name=null,
>> family=ubuntu, version=10.04, arch=null, is64Bit=true, description=Ubuntu
>> 10.04 LTS], state=RUNNING, loginPort=22,
>> hostname=humaninferencerackspace-d4b, privateAddresses=[10.183.34.3],
>> publicAddresses=[50.57.166.89], hardware=[id=4, providerId=4, name=2GB
>> server, processors=[[cores=8.0, speed=1.0]], ram=2048, volumes=[[id=null,
>> type=LOCAL, size=80.0, device=null, durable=true, isBootDevice=true]],
>> supportsImage=ALWAYS_TRUE, tags=[]], loginUser=root, userMetadata={},
>> tags=[]]}], configuration={fs.default.name=hdfs://
>> 50-57-166-89.static.cloud-ips.com:8020/, mapred.job.tracker=
>> 50-57-166-89.static.cloud-ips.com:8021, hadoop.job.ugi=root,root,
>> hadoop.rpc.socket.factory.class.default=org.apache.hadoop.net.SocksSocketFactory,
>> hadoop.socks.server=localhost:6666}}
>> You can log into instances using the following ssh commands:
>> 'ssh -i /home/hadoepje/.ssh/id_rsa -o "UserKnownHostsFile /dev/null" -o
>> StrictHostKeyChecking=no hadoepje@50.57.166.152'
>> 'ssh -i /home/hadoepje/.ssh/id_rsa -o "UserKnownHostsFile /dev/null" -o
>> StrictHostKeyChecking=no hadoepje@50.57.166.89'
>> hadoepje@hadoepgrote:~/whirr/whirr-0.7.1/bin$
>> /var/log/hadoop/logs/SecurityAuth.audit (Permission denied)
>> bash: syntax error near unexpected token `Permission'
>>
>>  ------------------------------
>> *From:* Paritosh Ranjan [pranjan@xebia.com]
>> *Sent:* Thursday, June 21, 2012 1:21 PM
>> *To:* user@whirr.apache.org
>> *Subject:* Whirr 0.7.1 : CDH : Rackspace : Problem
>>
>>    Hi,
>>
>> I am switching to Whirr 0.7.1 from 0.6-someversion
>> I am facing some issue while installing the cdh distribution on RackSpace
>> ( I am able to installing the Apache distribution ).
>>
>> I have followed the instructions on this page
>> http://whirr.apache.org/docs/0.7.1/quick-start-guide.html.
>>
>> This is my hadoop.properties file
>>
>> whirr.cluster-name=<theclustername>
>> whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1
>> hadoop-datanode+hadoop-tasktracker
>> whirr.provider=cloudservers-us
>> whirr.identity=<theidentity :)>
>> whirr.credential=<thepassword :)>
>> whirr.hardware-id=4
>> whirr.image=49
>> whirr.private-key-file=/home/hadoepje/.ssh/id_rsa
>> whirr.public-key-file=/home/hadoepje/.ssh/id_rsa.pub
>> whirr.hadoop.install-function=install_cdh_hadoop
>> whirr.hadoop.configure-function=configure_cdh_hadoop
>>
>>
>> However, I am getting this error. Can someone help?
>>
>> Successfully executed configure script: [output=        at
>> java.io.FileOutputStream.openAppend(Native Method)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:207)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
>>         at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
>>         at
>> org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
>>         at
>> org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:216)
>>         at
>> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
>>         at
>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
>> hadoop-0.20-namenode.
>> Safe mode is OFF
>> , error=12/06/21 10:58:38 INFO namenode.FSNamesystem:
>> supergroup=supergroup
>> 12/06/21 10:58:38 INFO namenode.FSNamesystem: isPermissionEnabled=true
>> 12/06/21 10:58:38 INFO namenode.FSNamesystem:
>> dfs.block.invalidate.limit=1000
>> 12/06/21 10:58:38 INFO namenode.FSNamesystem: isAccessTokenEnabled=false
>> accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
>> 12/06/21 10:58:38 INFO common.Storage: Image file of size 110 saved in 0
>> seconds.
>> 12/06/21 10:58:38 INFO common.Storage: Storage directory
>> /data/hadoop/hdfs/name has been successfully formatted.
>> 12/06/21 10:58:38 INFO namenode.NameNode: SHUTDOWN_MSG:
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down NameNode at
>> 50-57-166-89.static.cloud-ips.com/50.57.166.89
>> ************************************************************/
>> , exitCode=0]
>> Successfully executed configure script:
>> [output=java.io.FileNotFoundException:
>> /var/log/hadoop/logs/SecurityAuth.audit (Permission denied)
>>         at java.io.FileOutputStream.openAppend(Native Method)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:207)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
>>         at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
>>         at
>> org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
>>         at
>> org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:216)
>>         at
>> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
>>         at
>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
>> hadoop-0.20-tasktracker.
>>
>> Thanks and Regards,
>> Paritosh
>>
>
>

Mime
View raw message