jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Heneveld (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (JCLOUDS-1356) jclouds occasionally can't log in to AWS EC2 instance
Date Wed, 15 Nov 2017 16:45:00 GMT

    [ https://issues.apache.org/jira/browse/JCLOUDS-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16253762#comment-16253762

Alex Heneveld commented on JCLOUDS-1356:

Following instructions at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html
was able to hack our way in.

The {{authorized_keys}} file was empty on the box.  (On a healthy machine they keypair is

Looking further, in {{/var/log/messages}} we see:

{{/var/log/messages-20171115:Nov 15 11:22:05 localhost getsshkey: Fetching public key 'jclouds#brooklyn-ozgi6s-alex-ampcluster2-b0bz-nginx-jem6#1a'
from metadata server.../etc/rc.d/init.d/getsshkey: line 57: [: too many arguments}}

Whereas on a healthy machine it says {{Done}} after {{server...}}.

Looking at {{/etc/rc.d/init.d/getsshkey}} lines 56-57 are:

  code=`curl -s -L -w '%{http_code}' -o "/root/.ssh/${key_name}.pub" "${key_index}/openssh-key"`
  while [ $code -ge 500 ]; do

I'm not sure under what circumstances curl can return multiple words for {{ %{http_code} }}
but that seems to be the problem.  Normally it returns eg {{200}}; and {{-ge}} gives different
errors if {{$code}} is empty or a single non-numeric word.  The only way I could get the {{[:
too many arguments}} message happens if {{$code}} is something like {{0 1}} or {{a b}}.  By
the time I was able to get on manually to try I got {{200}} for {{$code}} so can't tell more
details of the cause.

My guess is the key isn't ready and {{curl}} returns something this script isn't expecting.


In any event this is a problem with the image, and the same problem would occur irrespective
of jclouds.  I'll close the issue here and report to Rightscale.

Workaround for us is to specify a known {{keyPair}} which might avoid the race, or use a different

> jclouds occasionally can't log in to AWS EC2 instance
> -----------------------------------------------------
>                 Key: JCLOUDS-1356
>                 URL: https://issues.apache.org/jira/browse/JCLOUDS-1356
>             Project: jclouds
>          Issue Type: Bug
>    Affects Versions: 2.0.2
>         Environment: Amazon, CentOS RightImage
>            Reporter: Alex Heneveld
> I call to create a machine in AWS with basic settings (unique key pair and security group),
and jclouds ultimately comes back failing with:
> {code}
> 2017-11-15T11:22:10,546 ERROR 107 j.compute [user thread 3] << problem customizing
> org.jclouds.rest.AuthorizationException: (root:rsa[fingerprint(fc:4f:57:c5:e6:db:00:f5:39:7b:f5:6b:1b:de:cf:fa),sha1(87:28:ea:42:e7:9f:6b:45:4a:ce:09:0a:30:02:46:ee:95:fa:b2:16)]@
error acquiring {hostAndPort=, loginUser=root, ssh=null, connectTimeout=60000,
sessionTimeout=60000} (out of retries - max 50): Exhausted available authentication methods
>         at org.jclouds.sshj.SshjSshClient.propagate(SshjSshClient.java:394) [108:jclouds-sshj:2.0.2]
>         at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:205) [108:jclouds-sshj:2.0.2]
>         at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:224) [108:jclouds-sshj:2.0.2]
>         at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:72)
>         at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:123)
>         at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:50)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745) [?:?]
> Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication
>         at net.schmizz.sshj.SSHClient.auth(SSHClient.java:217) ~[?:?]
>         at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:316) ~[?:?]
>         at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:335) ~[?:?]
>         at org.jclouds.sshj.SSHClientConnection.create(SSHClientConnection.java:163)
>         at org.jclouds.sshj.SSHClientConnection.create(SSHClientConnection.java:49) ~[?:?]
>         at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:195) ~[?:?]
>         ... 8 more
> {code}
> The machine _is_ created in AWS, and accessible on 22, just it doesn't like jclouds attempting
to log in.
> This is using imageId us-east-1/ami-5492ba3c - autodetected looking for a CentOS 7.0
machine from one of a set of trusted providers (including Rightscale who provide this).
> Most of the time it works fine.  It's <5% of such requests which fail.

This message was sent by Atlassian JIRA

View raw message