cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Yadav <rohit.ya...@shapeblue.com>
Subject Re: Cant add additional management servers with multiple IPs
Date Fri, 28 Aug 2020 07:01:27 GMT
Hi Adam, the mentioned bug seems to be fixed in a 4.11 release and you're on 4.13.1 so ideally
you shouldn't hit that.

The issue seems to be that the agents got certificates created with a root CA that is not
validated by the (additional) management servers. Some hints and checks you can perform:


  *   Did you add all the three management servers simultaneously?
  *   Can you restart all the management servers one by one and see if agents still fail to
connect
  *   To manually re-key the agents, you can set ca.plugin.root.auth.strictness global setting
to false (no need to restart the mgmt server) which will allow the agents to connect and then
using API or UI->Infra-> KVM hosts -> provision certificates again (or use API provisionCertificate
for hosts and cpvm/ssvm)
  *   Last resort, backup DB and delete the ca.plugin.root.public.key, ca.plugin.root.private.key,
ca.plugin.root.ca.certificate and stop all mgmt server, start one mgmt server and when it's
online start remaining. This will re-create root CA keypair and cert and perform the previous
step (change auth strictness to false and re-key the agents;

Hope this helps.


Regards.

________________________________
From: Adam Witwicki <awitwicki@oakfordis.com>
Sent: Monday, August 17, 2020 15:52
To: users@cloudstack.apache.org <users@cloudstack.apache.org>
Subject: Cant add additional management servers with multiple IPs

Hi Guys

Trying to set up cloudstack 4.13.1, but I am getting SSL cert errors on the 2 additional management
servers I'm trying to setup.
These servers have more than one IP - could it be related to this bug https://github.com/apache/cloudstack/issues/2530





Name        : cloudstack-management
Arch        : x86_64
Version     : 4.13.1.0
Release     : shapeblue0.el7

Error from 1st management server
2020-08-17 10:43:56,747 ERROR [o.a.c.c.p.RootCACustomTrustManager] (pool-60-thread-1:null)
(logid:) Certificate ownership verification failed for client: 10.10.216.221
2020-08-17 10:43:56,747 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-5:null) (logid:)
SSL error caught during wrap data: General SSLEngine problem, for local address=/10.10.216.200:8250,
remote address=/10.10.216.221:53568.
2020-08-17 10:43:56,797 ERROR [o.a.c.c.p.RootCACustomTrustManager] (pool-61-thread-1:null)
(logid:) Certificate ownership verification failed for client: 10.10.216.221
2020-08-17 10:43:56,798 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-5:null) (logid:)
SSL error caught during wrap data: General SSLEngine problem, for local address=/10.10.216.200:8250,
remote address=/10.10.216.221:53570.


Error from additional management server I'm trying to add
2020-08-17 10:43:56,640 ERROR [c.c.u.n.Link] (StatsCollector-2:ctx-aa7d0a75) (logid:10ec5992)
SSL error caught during unwrap data: Received fatal alert: certificate_unknown, for local
address=/10.10.216.221:53564, remote address=/10.10.216.200:8250. The client may have invalid
ca-certificates.
2020-08-17 10:43:56,641 WARN  [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-2:ctx-aa7d0a75)
(logid:10ec5992) Unable to connect to peer management server: 168482836, ip: 10.10.216.200
due to SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with peer management
server '168482836' on 10.10.216.200:8250
java.io.IOException: SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with
peer management server '168482836' on 10.10.216.200:8250
2020-08-17 10:43:56,641 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-2:ctx-aa7d0a75)
(logid:10ec5992) Seq 66-1928103590467993603: Unable to forward null
2020-08-17 10:43:56,641 WARN  [c.c.a.m.AgentManagerImpl] (StatsCollector-2:ctx-aa7d0a75) (logid:10ec5992)
Resource [Host:66] is unreachable: Host 66: Unable to reach the peer that the agent is connected
2020-08-17 10:43:56,641 WARN  [c.c.r.ResourceManagerImpl] (StatsCollector-2:ctx-aa7d0a75)
(logid:10ec5992) Unable to obtain host 66 statistics.
2020-08-17 10:43:56,641 WARN  [c.c.s.StatsCollector] (StatsCollector-2:ctx-aa7d0a75) (logid:10ec5992)
The Host stats is null for host: 66
2020-08-17 10:43:56,698 ERROR [c.c.u.n.Link] (StatsCollector-2:ctx-aa7d0a75) (logid:10ec5992)
SSL error caught during unwrap data: Received fatal alert: certificate_unknown, for local
address=/10.10.216.221:53566, remote address=/10.10.216.200:8250. The client may have invalid
ca-certificates.
2020-08-17 10:43:56,698 WARN  [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-2:ctx-aa7d0a75)
(logid:10ec5992) Unable to connect to peer management server: 168482836, ip: 10.10.216.200
due to SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with peer management
server '168482836' on 10.10.216.200:8250
java.io.IOException: SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with
peer management server '168482836' on 10.10.216.200:8250
2020-08-17 10:43:56,699 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-2:ctx-aa7d0a75)
(logid:10ec5992) Seq 69-2867104112774742021: Unable to forward null
2020-08-17 10:43:56,748 ERROR [c.c.u.n.Link] (StatsCollector-2:ctx-aa7d0a75) (logid:10ec5992)
SSL error caught during unwrap data: Received fatal alert: certificate_unknown, for local
address=/10.10.216.221:53568, remote address=/10.10.216.200:8250. The client may have invalid
ca-certificates.


I thought I solved this by following  http://mail-archives.apache.org/mod_mbox/cloudstack-users/201805.mbox/%3CVI1PR0701MB186911B8E6BA4B81E00EA963E9800@VI1PR0701MB1869.eurprd07.prod.outlook.com%3E



But when adding KVM agents I get this on the management server

address=/10.10.216.222:38570.
2020-08-17 11:18:13,195 ERROR [o.a.c.c.p.RootCACustomTrustManager] (pool-13-thread-1:null)
(logid:) Certificate ownership verification failed for client: 10.10.216.221
2020-08-17 11:18:13,196 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-1:null) (logid:)
SSL error caught during wrap data: General SSLEngine problem, for local address=/10.10.216.200:8250,
remote address=/10.10.216.221:33998.
2020-08-17 11:18:13,277 ERROR [o.a.c.c.p.RootCACustomTrustManager] (pool-14-thread-1:null)
(logid:) Certificate ownership verification failed for client: 10.10.216.221
2020-08-17 11:18:13,278 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-1:null) (logid:)
SSL error caught during wrap data: General SSLEngine problem, for local address=/10.10.216.200:8250,
remote address=/10.10.216.221:34000.



Any help appricated

Thanks

Adam

Disclaimer Notice:
This email has been sent by Oakford Technology Limited, while we have checked this e-mail
and any attachments for viruses, we can not guarantee that they are virus-free. You must therefore
take full responsibility for virus checking.
This message and any attachments are confidential and should only be read by those to whom
they are addressed. If you are not the intended recipient, please contact us, delete the message
from your computer and destroy any copies. Any distribution or copying without our prior permission
is prohibited.
Internet communications are not always secure and therefore Oakford Technology Limited does
not accept legal responsibility for this message. The recipient is responsible for verifying
its authenticity before acting on the contents. Any views or opinions presented are solely
those of the author and do not necessarily represent those of Oakford Technology Limited.
Registered address: Oakford Technology Limited, The Manor House, Potterne, Wiltshire. SN10
5PN.
Registered in England and Wales No. 5971519


rohit.yadav@shapeblue.comĀ 
www.shapeblue.com
3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
@shapeblue
  
 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message