whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Longbin Lai <longbin....@gmail.com>
Subject dfs configuration in whirr
Date Mon, 16 Jun 2014 07:54:02 GMT
Hi there,
Currently I am using whirr to deploy hadoop in Amazon EC2, and I have
encountered the issues as posted here,
http://mail-archives.apache.org/mod_mbox/whirr-user/201210.mbox/%3CCAHZL8y8+PNM_7uao-77vqczT+BCfbY_SvYxUprm=K+P9ngf4hQ@mail.gmail.com%3E.


The poster mentioned that we can have a larger hdfs configured capacity by
using store-instance instead. However, when I switch to a store-instance
image, the hdfs capacity remains the same and I do not see any link from
the /data0 directory to /mnt (i.e /data0 is simply a normal directory
created under / rather than a link to /mnt as the poster said). Further, I
try to configure the dfs directories in /mnt directly, while I cannot even
start the hadoop instances in this case which I believed that Hadoop donot
have the previledge to create the required direcotries there.

Below is the fragments of the configuration file of my whirr,
#################################################
whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,9
hadoop-datanode+hadoop-tasktracker
whirr.provider=aws-ec2
#whirr.identity=<Access Key Removed>
#whirr.credential=O/ICH7Hf6erSp0jiwC1W8fJBPMm7hsYNDImsdP2i
whirr.hardware-id=m3.xlarge
whirr.image-id=us-east-1/ami-30837058 # this is an store-instance image
whirr.location-id=us-east-1
...
hadoop-mapreduce.mapred.local.dir=/data0/hadoop/mapred/local
hadoop-mapreduce.mapred.system.dir=/data0/hadoop/system/mapred
...
#################################################
Can somebody help resolve this issue? Thank you very much.

-- 
Longbin Lai, phD Candidate,
Database Research Group
School of Computer Science and Engineering
University of New South Wales
NSW 2052, Sydney.

e-mail: llai@cse.unsw.edu.au <lxue@cse.unsw.edu.au>
main page: http://robeen.me

Mime
View raw message