cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Piotr Pisz" <pi...@piszki.pl>
Subject strange behavior ha
Date Tue, 17 Nov 2020 11:26:47 GMT
Hi Users!

I have a question asking for an explanation of the HA mechanism which behaves quite strangely

We have three mgmt servers, IP 10.89.12.101,102,103
And NFS Pool (sys2.cenagis.local) on floating IP via haproxy and nfs-ganesha.
We make a simple test, power off 10.89.12.101, and we saw strange infromation in cloudstack-agent
logs, NFS checks are going via cs mgmt IP and not via floating IP (sys2.cenagis.local), why?

Do we have a logical error somewhere?

Best regards,
Piotr

2020-11-17 09:56:37,140 DEBUG [kvm.resource.KVMHAChecker] (pool-72086-thread-1:null) (logid:0b27330b)
Executing while with timeout : 360000
2020-11-17 09:56:37,148 DEBUG [kvm.resource.KVMHAChecker] (pool-72086-thread-1:null) (logid:0b27330b)
Execution is successful.
2020-11-17 09:56:37,149 DEBUG [kvm.resource.KVMHAChecker] (pool-72086-thread-1:null) (logid:0b27330b)
KVMHAChecker pool: sys2.cenagis.local
2020-11-17 09:56:37,149 DEBUG [kvm.resource.KVMHAChecker] (pool-72086-thread-1:null) (logid:0b27330b)
KVMHAChecker result: null
2020-11-17 09:56:37,149 DEBUG [kvm.resource.KVMHAChecker] (pool-72086-thread-1:null) (logid:0b27330b)
KVMHAChecker parser: =====> ALIVE <=====
2020-11-17 09:56:37,149 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:0b27330b)
Seq 25-6380193297100515623:  { Ans: , MgmtId: 176206389408022, via: 25, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"Heart is beating...","wait":0}}]
}
2020-11-17 09:56:39,048 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:246f60a3)
Request:Seq 25-8191766247210225151:  { Cmd , MgmtId: 176206389407982, via: 25, Ver: v1, Flags:
100011, [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"27135d91-2859-3df8-9464-77f12c886598-LibvirtComputingResource","privateNetwork":{"ip":"10.89.12.101","netmask":"255.255.255.0","mac":"6a:fd:73:b6:d0:46","isSecurityGroupEnabled":false},"storageNetwork1":{"ip":"10.89.12.101","netmask":"255.255.255.0","mac":"6a:fd:73:b6:d0:46","isSecurityGroupEnabled":false}},"wait":20}}]
}
2020-11-17 09:56:39,048 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:246f60a3)
Processing command: com.cloud.agent.api.CheckOnHostCommand
2020-11-17 09:56:39,049 DEBUG [kvm.resource.KVMHAChecker] (pool-72087-thread-1:null) (logid:246f60a3)
Executing: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i sys2.cenagis.local
-p /HA -m /mnt/9cc142eb-ff79-3ea4-957d-9a45b987604e -h 10.89.12.101 -r -t 60
2020-11-17 09:56:39,049 DEBUG [kvm.resource.KVMHAChecker] (pool-72087-thread-1:null) (logid:246f60a3)
Executing while with timeout : 360000
2020-11-17 09:56:39,058 DEBUG [kvm.resource.KVMHAChecker] (pool-72087-thread-1:null) (logid:246f60a3)
Execution is successful.
2020-11-17 09:56:39,058 DEBUG [kvm.resource.KVMHAChecker] (pool-72087-thread-1:null) (logid:246f60a3)
KVMHAChecker pool: sys2.cenagis.local
2020-11-17 09:56:39,058 DEBUG [kvm.resource.KVMHAChecker] (pool-72087-thread-1:null) (logid:246f60a3)
KVMHAChecker result: null
2020-11-17 09:56:39,058 DEBUG [kvm.resource.KVMHAChecker] (pool-72087-thread-1:null) (logid:246f60a3)
KVMHAChecker parser: =====> DEAD <======
2020-11-17 09:56:39,058 DEBUG [kvm.resource.KVMHAChecker] (pool-72087-thread-1:null) (logid:246f60a3)
read heartbeat failed:
2020-11-17 09:56:39,058 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:246f60a3)
Seq 25-8191766247210225151:  { Ans: , MgmtId: 176206389407982, via: 25, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2020-11-17 09:56:52,834 DEBUG [kvm.resource.LibvirtConnection] (Thread-11215:null) (logid:)
Looking for libvirtd connection at: qemu:///system
2020-11-17 09:56:52,834 DEBUG [kvm.storage.IscsiStorageCleanupMonitor] (Thread-11215:null)
(logid:) found 0 domains
2020-11-17 09:56:58,860 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:4a826d3a)
Request:Seq 25-6380193297100515624:  { Cmd , MgmtId: 176206389408022, via: 25, Ver: v1, Flags:
100011, [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"de2a757a-4d20-39c1-97be-2eb0096c32fb-LibvirtComputingResource","privateNetwork":{"ip":"10.89.12.103","netmask":"255.255.255.0","mac":"7e:c2:06:20:06:48","isSecurityGroupEnabled":false},"storageNetwork1":{"ip":"10.89.12.103","netmask":"255.255.255.0","mac":"7e:c2:06:20:06:48","isSecurityGroupEnabled":false}},"wait":20}}]
}
2020-11-17 09:56:58,860 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:4a826d3a)
Processing command: com.cloud.agent.api.CheckOnHostCommand
2020-11-17 09:56:58,861 DEBUG [kvm.resource.KVMHAChecker] (pool-72088-thread-1:null) (logid:4a826d3a)
Executing: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i sys2.cenagis.local
-p /HA -m /mnt/9cc142eb-ff79-3ea4-957d-9a45b987604e -h 10.89.12.103 -r -t 60
2020-11-17 09:56:58,861 DEBUG [kvm.resource.KVMHAChecker] (pool-72088-thread-1:null) (logid:4a826d3a)
Executing while with timeout : 360000
2020-11-17 09:56:58,870 DEBUG [kvm.resource.KVMHAChecker] (pool-72088-thread-1:null) (logid:4a826d3a)
Execution is successful.
2020-11-17 09:56:58,870 DEBUG [kvm.resource.KVMHAChecker] (pool-72088-thread-1:null) (logid:4a826d3a)
KVMHAChecker pool: sys2.cenagis.local
2020-11-17 09:56:58,870 DEBUG [kvm.resource.KVMHAChecker] (pool-72088-thread-1:null) (logid:4a826d3a)
KVMHAChecker result: null
2020-11-17 09:56:58,870 DEBUG [kvm.resource.KVMHAChecker] (pool-72088-thread-1:null) (logid:4a826d3a)
KVMHAChecker parser: =====> ALIVE <=====
2020-11-17 09:56:58,870 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:4a826d3a)
Seq 25-6380193297100515624:  { Ans: , MgmtId: 176206389408022, via: 25, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"Heart is beating...","wait":0}}]
}


Mime
View raw message