whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: Problems using whirr to launch ubuntu machines
Date Tue, 18 Sep 2012 04:40:14 GMT
Were you able to find out anything additional about this Andrew?  I 
tried grep'ing through the jclouds code, but not being familiar with the 
code I wasn't able to zero in on where it determines whether an instance 
is running Ubuntu.

Thanks,

DR

On 09/17/2012 12:14 PM, Andrew Bayer wrote:
> Hmm - I'd guess it's the platform type, but there could be some other AMI
> flag - I'll check in the jclouds code.
>
> A.
>
> On Sun, Sep 16, 2012 at 10:07 PM, David Rosenstrauch <darose@darose.net>wrote:
>
>> OK, I think I have some clue what's happening here.
>>
>> When I launch my cluster using the stock Ubuntu AMI (e.g., ami-4fad7426)
>> everything works fine.  But when I launch using my own AMI which I've based
>> off of that, it fails.
>>
>> I think the reason is that Whirr/jclouds is able to determine that the
>> stock instance is running Ubuntu (and so login using the correct user name)
>> and it's not able to determine that about mine.
>>
>> Anyone know how Whirr/jclouds is determining that the OS type is Ubuntu?
>>   Is it the presence of the text "ubuntu" in the "source" attribute on EC2?
>>   (e.g., "099720109477/ubuntu/images/**hvm/ubuntu-precise-12.04-**amd64-server-2012042")
>>   My AMI doesn't mention "ubuntu" in its source/name.  Or is it because the
>> EC2 "platform" type is "Ubuntu Cloud Guest"?  (In my AMI, the platform type
>> appears as "Other Linux".)  Is there any way I can force Whirr/jclouds and
>> tell it that my AMI is based on Ubuntu?
>>
>> Thanks,
>>
>> DR
>>
>>
>> On 09/16/2012 11:48 PM, David Rosenstrauch wrote:
>>
>>> Seems to be same issue as this:
>>>
>>> https://github.com/jclouds/**jclouds/issues/748<https://github.com/jclouds/jclouds/issues/748>
>>>
>>> Doesn't look like there's any fix for this, near as I can tell.
>>>
>>>
>>> @Sebastian Schoenherr:  which version of Whirr are you using (and/or
>>> which Ubuntu AMI base) that you're able to successfully launch Ubuntu
>>> instances using whirr.login-user?
>>>
>>> Thanks,
>>>
>>> DR
>>>
>>> On 09/16/2012 11:34 PM, David Rosenstrauch wrote:
>>>
>>>> Hmmmm ... no luck.  I've tried now with whirr.bootstrap-user,
>>>> whirr.login-user (which whirr tells me is now deprecated in favor of
>>>> whirr.bootstrap-user), and with no user at all.  Result is always the
>>>> same:  instances start up, but whirr fails to initialize them.  The
>>>> whirr log shows the following error message (which apparently indicates
>>>> an ssh login failure):
>>>>
>>>> java.lang.**IllegalStateException: Invalid packet: indicated length
>>>> 1349281121 too large
>>>>
>>>> And on the nodes, I keep seeing Ubuntu starting up a process that echo's
>>>> the following message:  Please login as the user "ubuntu" rather than
>>>> the user "root".
>>>>
>>>> So it seems that somehow whirr keeps trying to login as root and
>>>> failing.  Not sure why.
>>>>
>>>>
>>>> The Ubuntu AMI I'm using is based off of one provided by the Ubuntu
>>>> project itself.  Specifically ami-4fad7426.  (See this page for more
>>>> details:  http://cloud.ubuntu.com/ami/)
>>>>
>>>> Anyone have any idea what might be going wrong here?
>>>>
>>>> Thanks,
>>>>
>>>> DR
>>>>
>>>> On 09/16/2012 03:48 PM, Sebastian Schoenherr wrote:
>>>>
>>>>> Hi David,
>>>>> try to set: whirr.login-user=ubuntu and skip the bootstrap-user.
>>>>> works for me,
>>>>> Chers Sebsatian
>>>>>
>>>>> On 16/09/2012 16:41, David Rosenstrauch wrote:
>>>>>
>>>>>> Whoops sorry - didn't understand.  Props file posted below  Again,
>>>>>> this is using Whirr v0.8.
>>>>>>
>>>>>> DR
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> whirr.env.repo=cdh4
>>>>>> whirr.cluster-name=hadoopcc
>>>>>> whirr.instance-templates=1 hadoop-namenode+hadoop-**jobtracker,8
>>>>>> hadoop-datanode+hadoop-**tasktracker
>>>>>> whirr.instance-templates-**minimum-number-of-instances=1
>>>>>> hadoop-namenode+hadoop-**jobtracker, 6 hadoop-datanode+hadoop-**
>>>>>> tasktracker
>>>>>> whirr.max-startup-retries=4
>>>>>> whirr.provider=aws-ec2
>>>>>> whirr.identity=...
>>>>>> whirr.credential=...
>>>>>> whirr.private-key-file=${sys:**user.home}/.ssh/id_rsa
>>>>>> whirr.public-key-file=${sys:**user.home}/.ssh/id_rsa.pub
>>>>>> whirr.hadoop.install-function=**install_cdh_hadoop
>>>>>> whirr.hadoop.configure-**function=configure_cdh_hadoop
>>>>>> whirr.hardware-id=cc1.4xlarge
>>>>>> whirr.location-id=us-east-1
>>>>>> whirr.image-id=us-east-1/ami-**dd7fcfb4
>>>>>> whirr.bootstrap-user=ubuntu
>>>>>> hadoop-hdfs.dfs.replication=2
>>>>>> hadoop-mapreduce.mapred.**tasktracker.map.tasks.maximum=**8
>>>>>> hadoop-mapreduce.mapred.**tasktracker.reduce.tasks.**maximum=8
>>>>>> hadoop-mapreduce.mapred.**reduce.tasks=64
>>>>>> hadoop-mapreduce.mapred.task.**timeout=1800000
>>>>>> hadoop-mapreduce.mapred.child.**java.opts=-Xmx1024m
>>>>>>
>>>>>>
>>>>>> On 09/15/2012 04:20 PM, Alex Heneveld wrote:
>>>>>>
>>>>>>> Hi David,
>>>>>>>
>>>>>>> I think Andrei is asking if you can send the props/config file
>>>>>>> (recipe) you
>>>>>>> are using.
>>>>>>>
>>>>>>> Best,
>>>>>>> Alex
>>>>>>> On Sep 15, 2012 7:43 PM, "David Rosenstrauch" <darose@darose.net>
>>>>>>> wrote:
>>>>>>>
>>>>>>>   We're using whirr v0.8.
>>>>>>>>
>>>>>>>> What message is it that you're suggesting I add to the
>>>>>>>> hadoop.properties
>>>>>>>> file?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> DR
>>>>>>>>
>>>>>>>> On Sat, September 15, 2012 12:59 pm, Andrei Savu wrote:
>>>>>>>>
>>>>>>>>> Please add to this message your .properties file. What
version of
>>>>>>>>> Whirr
>>>>>>>>> are
>>>>>>>>> you using?
>>>>>>>>> On Sep 15, 2012 7:52 PM, "David Rosenstrauch" <darose@darose.net>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>   We've been using whirr to launch instances of the (redhat-based)
>>>>>>>>>> Amazon
>>>>>>>>>> AMI, which has been working.  However, I'm now trying
to switch to
>>>>>>>>>> launching an ubuntu-based AMI, and I'm running into
problems.
>>>>>>>>>>
>>>>>>>>>> Ubuntu, as you might recall, doesn't allow you to
log in
>>>>>>>>>> directly as
>>>>>>>>>> root.
>>>>>>>>>>    Rather, you log in as user ubuntu, and then sudo
to root.
>>>>>>>>>> This is
>>>>>>>>>> causing problems when whirr tries to set up my data
nodes.
>>>>>>>>>>
>>>>>>>>>> I've configured whirr to use "ubuntu" as the bootstrap
user, which
>>>>>>>>>> seems
>>>>>>>>>> to work fine initially.  But after a short while
in the
>>>>>>>>>> installation
>>>>>>>>>> process, it fails.  And when I go onto the data node
machine to
>>>>>>>>>> watch
>>>>>>>>>> what's happening, I see that something (most likely
whirr) is
>>>>>>>>>> trying to
>>>>>>>>>> login as root.  So I'm fairly certain that's what's
causing the
>>>>>>>>>> installation to fail.
>>>>>>>>>>
>>>>>>>>>> Is there a way around this issue?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> DR


Mime
View raw message