whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: Problems using whirr to launch ubuntu machines
Date Tue, 18 Sep 2012 04:46:22 GMT
Hmmm .... I guess I'll have to consider either trying to use a stock AMI 
or relaxing the root login restriction.

Root logins are obviously do-able.  (Though not desirable from a 
security standpoint).

As far as the stock AMI ... not sure if that's do-able or not.  We need 
to create a custom node, which has some of our own software and data 
pre-installed before launch.  So I could only use a stock AMI if there's 
a way for me to command Whirr to automatically transfer a tarball to 
each new node, and then unzip it, and execute an install script.  Is 
there a way to instruct Whirr to do this?  I didn't see anything on the 
web site that described how this could be done.

Thanks,

DR

On 09/17/2012 12:51 PM, Alex Heneveld wrote:
>
> Guys,
>
> I'm not seeing any easy way to set this.  I've never stumbled across the
> option to set the AMI platform type (is this inferred from kernel ID?).
> In any case I think jclouds uses the imageLocation field compared with
> known patterns used by known image providers (in
> AWSEC2ReviseParsedImage) -- platform doesn't seem to be in the wire log.
>
> If using raw jclouds you can provide your own AdminAccess setup, that's
> the way to log in as ubuntu so I think the "solution" here would be to
> extend Whirr to expose this.
>
> Alternatively ... in this case ... is it an option either to use a stock
> AMI or permit root logins?  A stock AMI has the advantage that others
> will be using and testing against it.
>
> --A
>
>
> On 17/09/2012 17:16, allen wrote:
>>
>> Andrew Bayer <andrew.bayer@gmail.com> wrote:
>> Hmm - I'd guess it's the platform type, but there could be some other
>> AMI flag - I'll check in the jclouds code.
>>
>> A.
>>
>> On Sun, Sep 16, 2012 at 10:07 PM, David Rosenstrauch
>> <darose@darose.net <mailto:darose@darose.net>> wrote:
>>
>>     OK, I think I have some clue what's happening here.
>>
>>     When I launch my cluster using the stock Ubuntu AMI (e.g.,
>>     ami-4fad7426) everything works fine.  But when I launch using my
>>     own AMI which I've based off of that, it fails.
>>
>>     I think the reason is that Whirr/jclouds is able to determine that
>>     the stock instance is running Ubuntu (and so login using the
>>     correct user name) and it's not able to determine that about mine.
>>
>>     Anyone know how Whirr/jclouds is determining that the OS type is
>>     Ubuntu?  Is it the presence of the text "ubuntu" in the "source"
>>     attribute on EC2?  (e.g.,
>>
>> "099720109477/ubuntu/images/hvm/ubuntu-precise-12.04-amd64-server-2012042")
>>
>>      My AMI doesn't mention "ubuntu" in its source/name.  Or is it
>>     because the EC2 "platform" type is "Ubuntu Cloud Guest"?  (In my
>>     AMI, the platform type appears as "Other Linux".)  Is there any
>>     way I can force Whirr/jclouds and tell it that my AMI is based on
>>     Ubuntu?
>>
>>     Thanks,
>>
>>     DR
>>
>>
>>     On 09/16/2012 11:48 PM, David Rosenstrauch wrote:
>>
>>         Seems to be same issue as this:
>>
>>         https://github.com/jclouds/jclouds/issues/748
>>
>>         Doesn't look like there's any fix for this, near as I can tell.
>>
>>
>>         @Sebastian Schoenherr:  which version of Whirr are you using
>>         (and/or
>>         which Ubuntu AMI base) that you're able to successfully launch
>>         Ubuntu
>>         instances using whirr.login-user?
>>
>>         Thanks,
>>
>>         DR
>>
>>         On 09/16/2012 11:34 PM, David Rosenstrauch wrote:
>>
>>             Hmmmm ... no luck.  I've tried now with whirr.bootstrap-user,
>>             whirr.login-user (which whirr tells me is now deprecated
>>             in favor of
>>             whirr.bootstrap-user), and with no user at all.  Result is
>>             always the
>>             same:  instances start up, but whirr fails to initialize
>>             them.  The
>>             whirr log shows the following error message (which
>>             apparently indicates
>>             an ssh login failure):
>>
>>             java.lang.IllegalStateException: Invalid packet: indicated
>>             length
>>             1349281121 too large
>>
>>             And on the nodes, I keep seeing Ubuntu starting up a
>>             process that echo's
>>             the following message:  Please login as the user "ubuntu"
>>             rather than
>>             the user "root".
>>
>>             So it seems that somehow whirr keeps trying to login as
>>             root and
>>             failing.  Not sure why.
>>
>>
>>             The Ubuntu AMI I'm using is based off of one provided by
>>             the Ubuntu
>>             project itself.  Specifically ami-4fad7426.  (See this
>>             page for more
>>             details: http://cloud.ubuntu.com/ami/)
>>
>>             Anyone have any idea what might be going wrong here?
>>
>>             Thanks,
>>
>>             DR
>>
>>             On 09/16/2012 03:48 PM, Sebastian Schoenherr wrote:
>>
>>                 Hi David,
>>                 try to set: whirr.login-user=ubuntu and skip the
>>                 bootstrap-user.
>>                 works for me,
>>                 Chers Sebsatian
>>
>>                 On 16/09/2012 16:41, David Rosenstrauch wrote:
>>
>>                     Whoops sorry - didn't understand.  Props file
>>                     posted below  Again,
>>                     this is using Whirr v0.8.
>>
>>                     DR
>>
>>                     ---
>>
>>                     whirr.env.repo=cdh4
>>                     whirr.cluster-name=hadoopcc
>>                     whirr.instance-templates=1
>>                     hadoop-namenode+hadoop-jobtracker,8
>>                     hadoop-datanode+hadoop-tasktracker
>>
>> whirr.instance-templates-minimum-number-of-instances=1
>>                     hadoop-namenode+hadoop-jobtracker, 6
>>                     hadoop-datanode+hadoop-tasktracker
>>                     whirr.max-startup-retries=4
>>                     whirr.provider=aws-ec2
>>                     whirr.identity=...
>>                     whirr.credential=...
>>                     whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
>>
>> whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub
>>                     whirr.hadoop.install-function=install_cdh_hadoop
>>                     whirr.hadoop.configure-function=configure_cdh_hadoop
>>                     whirr.hardware-id=cc1.4xlarge
>>                     whirr.location-id=us-east-1
>>                     whirr.image-id=us-east-1/ami-dd7fcfb4
>>                     whirr.bootstrap-user=ubuntu
>>                     hadoop-hdfs.dfs.replication=2
>>
>> hadoop-mapreduce.mapred.tasktracker.map.tasks.maximum=8
>>
>> hadoop-mapreduce.mapred.tasktracker.reduce.tasks.maximum=8
>>                     hadoop-mapreduce.mapred.reduce.tasks=64
>>                     hadoop-mapreduce.mapred.task.timeout=1800000
>>                     hadoop-mapreduce.mapred.child.java.opts=-Xmx1024m
>>
>>
>>                     On 09/15/2012 04:20 PM, Alex Heneveld wrote:
>>
>>                         Hi David,
>>
>>                         I think Andrei is asking if you can send the
>>                         props/config file
>>                         (recipe) you
>>                         are using.
>>
>>                         Best,
>>                         Alex
>>                         On Sep 15, 2012 7:43 PM, "David Rosenstrauch"
>>                         <darose@darose.net <mailto:darose@darose.net>>
>>                         wrote:
>>
>>                             We're using whirr v0.8.
>>
>>                             What message is it that you're suggesting
>>                             I add to the
>>                             hadoop.properties
>>                             file?
>>
>>                             Thanks,
>>
>>                             DR
>>
>>                             On Sat, September 15, 2012 12:59 pm,
>>                             Andrei Savu wrote:
>>
>>                                 Please add to this message your
>>                                 .properties file. What version of
>>                                 Whirr
>>                                 are
>>                                 you using?
>>                                 On Sep 15, 2012 7:52 PM, "David
>>                                 Rosenstrauch" <darose@darose.net
>>                                 <mailto:darose@darose.net>>
>>                                 wrote:
>>
>>                                     We've been using whirr to launch
>>                                     instances of the (redhat-based)
>>                                     Amazon
>>                                     AMI, which has been working.
>>                                      However, I'm now trying to switch to
>>                                     launching an ubuntu-based AMI, and
>>                                     I'm running into problems.
>>
>>                                     Ubuntu, as you might recall,
>>                                     doesn't allow you to log in
>>                                     directly as
>>                                     root.
>>                                       Rather, you log in as user
>>                                     ubuntu, and then sudo to root.
>>                                     This is
>>                                     causing problems when whirr tries
>>                                     to set up my data nodes.
>>
>>                                     I've configured whirr to use
>>                                     "ubuntu" as the bootstrap user, which
>>                                     seems
>>                                     to work fine initially.  But after
>>                                     a short while in the
>>                                     installation
>>                                     process, it fails.  And when I go
>>                                     onto the data node machine to
>>                                     watch
>>                                     what's happening, I see that
>>                                     something (most likely whirr) is
>>                                     trying to
>>                                     login as root.  So I'm fairly
>>                                     certain that's what's causing the
>>                                     installation to fail.
>>
>>                                     Is there a way around this issue?
>>
>>                                     Thanks,
>>
>>                                     DR
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
>


Mime
View raw message