whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Bayer <andrew.ba...@gmail.com>
Subject Re: Problems using whirr to launch ubuntu machines
Date Mon, 17 Sep 2012 16:14:46 GMT
Hmm - I'd guess it's the platform type, but there could be some other AMI
flag - I'll check in the jclouds code.

A.

On Sun, Sep 16, 2012 at 10:07 PM, David Rosenstrauch <darose@darose.net>wrote:

> OK, I think I have some clue what's happening here.
>
> When I launch my cluster using the stock Ubuntu AMI (e.g., ami-4fad7426)
> everything works fine.  But when I launch using my own AMI which I've based
> off of that, it fails.
>
> I think the reason is that Whirr/jclouds is able to determine that the
> stock instance is running Ubuntu (and so login using the correct user name)
> and it's not able to determine that about mine.
>
> Anyone know how Whirr/jclouds is determining that the OS type is Ubuntu?
>  Is it the presence of the text "ubuntu" in the "source" attribute on EC2?
>  (e.g., "099720109477/ubuntu/images/**hvm/ubuntu-precise-12.04-**amd64-server-2012042")
>  My AMI doesn't mention "ubuntu" in its source/name.  Or is it because the
> EC2 "platform" type is "Ubuntu Cloud Guest"?  (In my AMI, the platform type
> appears as "Other Linux".)  Is there any way I can force Whirr/jclouds and
> tell it that my AMI is based on Ubuntu?
>
> Thanks,
>
> DR
>
>
> On 09/16/2012 11:48 PM, David Rosenstrauch wrote:
>
>> Seems to be same issue as this:
>>
>> https://github.com/jclouds/**jclouds/issues/748<https://github.com/jclouds/jclouds/issues/748>
>>
>> Doesn't look like there's any fix for this, near as I can tell.
>>
>>
>> @Sebastian Schoenherr:  which version of Whirr are you using (and/or
>> which Ubuntu AMI base) that you're able to successfully launch Ubuntu
>> instances using whirr.login-user?
>>
>> Thanks,
>>
>> DR
>>
>> On 09/16/2012 11:34 PM, David Rosenstrauch wrote:
>>
>>> Hmmmm ... no luck.  I've tried now with whirr.bootstrap-user,
>>> whirr.login-user (which whirr tells me is now deprecated in favor of
>>> whirr.bootstrap-user), and with no user at all.  Result is always the
>>> same:  instances start up, but whirr fails to initialize them.  The
>>> whirr log shows the following error message (which apparently indicates
>>> an ssh login failure):
>>>
>>> java.lang.**IllegalStateException: Invalid packet: indicated length
>>> 1349281121 too large
>>>
>>> And on the nodes, I keep seeing Ubuntu starting up a process that echo's
>>> the following message:  Please login as the user "ubuntu" rather than
>>> the user "root".
>>>
>>> So it seems that somehow whirr keeps trying to login as root and
>>> failing.  Not sure why.
>>>
>>>
>>> The Ubuntu AMI I'm using is based off of one provided by the Ubuntu
>>> project itself.  Specifically ami-4fad7426.  (See this page for more
>>> details:  http://cloud.ubuntu.com/ami/)
>>>
>>> Anyone have any idea what might be going wrong here?
>>>
>>> Thanks,
>>>
>>> DR
>>>
>>> On 09/16/2012 03:48 PM, Sebastian Schoenherr wrote:
>>>
>>>> Hi David,
>>>> try to set: whirr.login-user=ubuntu and skip the bootstrap-user.
>>>> works for me,
>>>> Chers Sebsatian
>>>>
>>>> On 16/09/2012 16:41, David Rosenstrauch wrote:
>>>>
>>>>> Whoops sorry - didn't understand.  Props file posted below  Again,
>>>>> this is using Whirr v0.8.
>>>>>
>>>>> DR
>>>>>
>>>>> ---
>>>>>
>>>>> whirr.env.repo=cdh4
>>>>> whirr.cluster-name=hadoopcc
>>>>> whirr.instance-templates=1 hadoop-namenode+hadoop-**jobtracker,8
>>>>> hadoop-datanode+hadoop-**tasktracker
>>>>> whirr.instance-templates-**minimum-number-of-instances=1
>>>>> hadoop-namenode+hadoop-**jobtracker, 6 hadoop-datanode+hadoop-**
>>>>> tasktracker
>>>>> whirr.max-startup-retries=4
>>>>> whirr.provider=aws-ec2
>>>>> whirr.identity=...
>>>>> whirr.credential=...
>>>>> whirr.private-key-file=${sys:**user.home}/.ssh/id_rsa
>>>>> whirr.public-key-file=${sys:**user.home}/.ssh/id_rsa.pub
>>>>> whirr.hadoop.install-function=**install_cdh_hadoop
>>>>> whirr.hadoop.configure-**function=configure_cdh_hadoop
>>>>> whirr.hardware-id=cc1.4xlarge
>>>>> whirr.location-id=us-east-1
>>>>> whirr.image-id=us-east-1/ami-**dd7fcfb4
>>>>> whirr.bootstrap-user=ubuntu
>>>>> hadoop-hdfs.dfs.replication=2
>>>>> hadoop-mapreduce.mapred.**tasktracker.map.tasks.maximum=**8
>>>>> hadoop-mapreduce.mapred.**tasktracker.reduce.tasks.**maximum=8
>>>>> hadoop-mapreduce.mapred.**reduce.tasks=64
>>>>> hadoop-mapreduce.mapred.task.**timeout=1800000
>>>>> hadoop-mapreduce.mapred.child.**java.opts=-Xmx1024m
>>>>>
>>>>>
>>>>> On 09/15/2012 04:20 PM, Alex Heneveld wrote:
>>>>>
>>>>>> Hi David,
>>>>>>
>>>>>> I think Andrei is asking if you can send the props/config file
>>>>>> (recipe) you
>>>>>> are using.
>>>>>>
>>>>>> Best,
>>>>>> Alex
>>>>>> On Sep 15, 2012 7:43 PM, "David Rosenstrauch" <darose@darose.net>
>>>>>> wrote:
>>>>>>
>>>>>>  We're using whirr v0.8.
>>>>>>>
>>>>>>> What message is it that you're suggesting I add to the
>>>>>>> hadoop.properties
>>>>>>> file?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> DR
>>>>>>>
>>>>>>> On Sat, September 15, 2012 12:59 pm, Andrei Savu wrote:
>>>>>>>
>>>>>>>> Please add to this message your .properties file. What version
of
>>>>>>>> Whirr
>>>>>>>> are
>>>>>>>> you using?
>>>>>>>> On Sep 15, 2012 7:52 PM, "David Rosenstrauch" <darose@darose.net>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>  We've been using whirr to launch instances of the (redhat-based)
>>>>>>>>> Amazon
>>>>>>>>> AMI, which has been working.  However, I'm now trying
to switch to
>>>>>>>>> launching an ubuntu-based AMI, and I'm running into problems.
>>>>>>>>>
>>>>>>>>> Ubuntu, as you might recall, doesn't allow you to log
in
>>>>>>>>> directly as
>>>>>>>>> root.
>>>>>>>>>   Rather, you log in as user ubuntu, and then sudo to
root.
>>>>>>>>> This is
>>>>>>>>> causing problems when whirr tries to set up my data nodes.
>>>>>>>>>
>>>>>>>>> I've configured whirr to use "ubuntu" as the bootstrap
user, which
>>>>>>>>> seems
>>>>>>>>> to work fine initially.  But after a short while in the
>>>>>>>>> installation
>>>>>>>>> process, it fails.  And when I go onto the data node
machine to
>>>>>>>>> watch
>>>>>>>>> what's happening, I see that something (most likely whirr)
is
>>>>>>>>> trying to
>>>>>>>>> login as root.  So I'm fairly certain that's what's causing
the
>>>>>>>>> installation to fail.
>>>>>>>>>
>>>>>>>>> Is there a way around this issue?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> DR
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message