cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Natalia Costas Lago <nata...@cesga.es>
Subject agent does not connect properly, Another active pool with the same uuid already exists
Date Tue, 03 Jan 2017 08:17:10 GMT

Hi,

One of our agents cannot connect to the server anymore... The manager 
complains about active pools already created. We rebooted the agent, but 
same result, no success.

Any hints about how to solve this without reinstalling the agent?

** Agent:

[root@c11-38 ~]# virsh pool-list
  Name                 State      Autostart
-------------------------------------------
  691f4f70-2724-4f9f-8537-6e0cc006dc41 active     no
  c69fb76e-f2f9-359e-bda0-1259786717af active     no

2017-01-03 09:04:44,654 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) 
(logid:) Connected to the server
2017-01-03 09:04:50,132 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) 
(logid:) Proccess agent startup answer, agent id = 0
2017-01-03 09:04:50,133 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) 
(logid:) Set agent id 0
2017-01-03 09:04:50,133 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) 
(logid:) Startup Response Received: agent id = 0
2017-01-03 09:04:50,487 INFO  [kvm.storage.LibvirtStorageAdaptor] 
(agentRequest-Handler-4:null) (logid:0fee19e8) Attempting to create 
storage pool c69fb76e-f2f9-359e-bda0-1259786717af (Gluster) in libvirt
2017-01-03 09:04:50,488 INFO  [kvm.storage.LibvirtStorageAdaptor] 
(agentRequest-Handler-4:null) (logid:0fee19e8) Found existing defined 
storage pool c69fb76e-f2f9-359e-bda0-1259786717af, using it.
2017-01-03 09:04:50,488 INFO  [kvm.storage.LibvirtStorageAdaptor] 
(agentRequest-Handler-4:null) (logid:0fee19e8) Trying to fetch storage 
pool c69fb76e-f2f9-359e-bda0-1259786717af from libvirt
2017-01-03 09:04:55,674 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) 
(logid:) Lost connection to the server. Dealing with the remaining 
commands...
2017-01-03 09:04:55,675 INFO  [utils.nio.NioClient] 
(Agent-Handler-2:null) (logid:) NioClient connection closed
2017-01-03 09:04:55,675 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) 
(logid:) Reconnecting...
2017-01-03 09:04:55,675 INFO  [utils.nio.NioClient] 
(Agent-Handler-2:null) (logid:) Connecting to 10.112.0.100:8250
2017-01-03 09:04:55,762 INFO  [utils.nio.NioClient] 
(Agent-Handler-2:null) (logid:) SSL: Handshake done
2017-01-03 09:04:55,762 INFO  [utils.nio.NioClient] 
(Agent-Handler-2:null) (logid:) Connected to 10.112.0.100:8250
2017-01-03 09:04:55,846 INFO  [kvm.storage.LibvirtStorageAdaptor] 
(Agent-Handler-1:null) (logid:) Attempting to create storage pool 
691f4f70-2724-4f9f-8537-6e0cc006dc41 (Filesystem) in libvirt
2017-01-03 09:04:55,846 INFO  [kvm.storage.LibvirtStorageAdaptor] 
(Agent-Handler-1:null) (logid:) Found existing defined storage pool 
691f4f70-2724-4f9f-8537-6e0cc006dc41, using it.
2017-01-03 09:04:55,846 INFO  [kvm.storage.LibvirtStorageAdaptor] 
(Agent-Handler-1:null) (logid:) Trying to fetch storage pool 
691f4f70-2724-4f9f-8537-6e0cc006dc41 from libvirt
2017-01-03 09:04:57,721 INFO  [cloud.agent.Agent] 
(AgentShutdownThread:null) (logid:) Stopping the agent: Reason = sig.kill

** Manager

2017-01-02 23:16:01,613 DEBUG [c.c.s.StorageManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Adding pool 
null to  host 1
2017-01-02 23:16:01,616 DEBUG [c.c.a.t.Request] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Seq 
1-5971210155939856387: Sending  { Cmd , MgmtId: 2199198367844, via: 
1(c11-38.lan.cesga.es), Ver: v1, Flags: 100011, 
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":5,"uuid":"c69fb76e-f2f9-359e-bda0-1259786717af","host":"cloudstack-storage.lan.cesga.es","path":"/primary","port":24007,"type":"Gluster"},"localPath":"/mnt//c69fb76e-f2f9-359e-bda0-1259786717af","wait":0}}]

}
2017-01-02 23:16:01,691 DEBUG [c.c.a.t.Request] 
(AgentManager-Handler-10:null) (logid:) Seq 1-5971210155939856386: 
Processing:  { Ans: , MgmtId: 2199198367844, via: 1, Ver: v1, Flags: 10, 
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2017-01-02 23:16:01,691 DEBUG [c.c.a.t.Request] 
(AgentManager-Handler-10:null) (logid:) Seq 1-5971210155939856387: 
Processing:  { Ans: , MgmtId: 2199198367844, via: 1, Ver: v1, Flags: 10, 
[{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"cloudstack-storage.lan.cesga.es","localPath":"/mnt//c69fb76e-f2f9-359e-bda0-1259786717af","hostPath":"/primary","poolType":"Gluster","capacityBytes":7188486029312,"availableBytes":7161281118208},"templateInfo":{},"result":true,"wait":0}}]

}
2017-01-02 23:16:01,691 DEBUG [c.c.a.t.Request] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Seq 
1-5971210155939856387: Received:  { Ans: , MgmtId: 2199198367844, via: 
1(c11-38.lan.cesga.es), Ver: v1, Flags: 10, { ModifyStoragePoolAnswer } }
2017-01-02 23:16:01,702 INFO  [o.a.c.s.d.p.DefaultHostListener] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Connection 
established between storage pool 
org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@2dee5c66 
and host 1
2017-01-02 23:16:01,703 DEBUG [c.c.s.StorageManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Found storage 
pool almacenamiento-primario-gluster of type Gluster
2017-01-02 23:16:01,703 DEBUG [c.c.s.StorageManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Total over 
provisioned capacity of the pool almacenamiento-primario-gluster id: 5 
is 7188486029312
2017-01-02 23:16:01,703 DEBUG [c.c.s.StorageManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Successfully 
set Capacity - 7188486029312 for capacity type - 3 , DataCenterId - 1, 
HostOrPoolId - 5, PodId null
2017-01-02 23:16:01,703 DEBUG [c.c.a.m.AgentManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Sending 
Connect to listener: ClusteredVirtualMachineManagerImpl
2017-01-02 23:16:01,703 DEBUG [c.c.v.VirtualMachineManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Received 
startup command from hypervisor host. host id: 1
2017-01-02 23:16:01,703 INFO [c.c.v.VirtualMachinePowerStateSyncImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Reset VM power 
state sync for host: 1
2017-01-02 23:16:01,747 DEBUG [c.c.a.t.Request] 
(AgentManager-Handler-2:null) (logid:) Seq 1-5971210155939856386: 
Processing:  { Ans: , MgmtId: 2199198367844, via: 1, Ver: v1, Flags: 10, 
[{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] }
2017-01-02 23:16:01,751 DEBUG [c.c.a.m.AgentManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Sending 
Connect to listener: ConsoleProxyListener
2017-01-02 23:16:01,752 DEBUG [c.c.a.m.AgentManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Sending 
Connect to listener: StorageCapacityListener
2017-01-02 23:16:01,752 DEBUG [c.c.a.m.AgentManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Sending 
Connect to listener: ComputeCapacityListener
2017-01-02 23:16:01,755 DEBUG [c.c.c.CapacityManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Found 1 VMs on 
host 1
2017-01-02 23:16:01,759 DEBUG [c.c.c.CapacityManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Found 0 VM, 
not running on host 1
2017-01-02 23:16:01,760 DEBUG [c.c.c.CapacityManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) No need to 
calibrate cpu capacity, host:1 usedCpu: 500 reservedCpu: 0
2017-01-02 23:16:01,760 DEBUG [c.c.c.CapacityManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) No need to 
calibrate memory capacity, host:1 usedMem: 1073741824 reservedMem: 0
2017-01-02 23:16:01,791 DEBUG [c.c.c.ConsoleProxyManagerImpl] 
(consoleproxy-1:ctx-23b32966) (logid:c300fd4b) Zone 1 is ready to launch 
console proxy
2017-01-02 23:16:01,809 DEBUG [c.c.a.m.AgentManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Sending 
Connect to listener: LocalStoragePoolListener
2017-01-02 23:16:01,811 DEBUG 
[o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) createPool 
Params @ scheme - Filesystem storageHost - 10.112.11.38 hostPath - 
/var/lib/libvirt/images port - -1
2017-01-02 23:16:01,811 DEBUG 
[o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Another active 
pool with the same uuid already exists
2017-01-02 23:16:01,812 WARN  [c.c.s.StorageManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Unable to 
setup the local storage pool for Host[-1-Routing]
com.cloud.utils.exception.CloudRuntimeException: Another active pool 
with the same uuid already exists
         at 
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.initialize(CloudStackPrimaryDataStoreLifeCycleImpl.java:341)
         at 
com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManagerImpl.java:607)
         at 
com.cloud.storage.LocalStoragePoolListener.processConnect(LocalStoragePoolListener.java:88)
         at 
com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:564)
         at 
com.cloud.agent.manager.AgentManagerImpl.handleConnectedAgent(AgentManagerImpl.java:1087)
         at 
com.cloud.agent.manager.AgentManagerImpl.access$000(AgentManagerImpl.java:120)
         at 
com.cloud.agent.manager.AgentManagerImpl$HandleAgentConnectTask.runInContext(AgentManagerImpl.java:1171)
         at 
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
         at 
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
         at java.lang.Thread.run(Thread.java:745)
2017-01-02 23:16:01,812 INFO  [c.c.u.e.CSExceptionErrorCode] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Could not find 
exception: com.cloud.exception.ConnectionException in error code list 
for exceptions
2017-01-02 23:16:01,812 WARN  [c.c.a.m.AgentManagerImpl] 
(AgentConnectTaskPool-1208:ctx-b95fb6f1) (logid:3c2b146e) Monitor 
LocalStoragePoolListener says there is an error in the connect process 
for 1 due to Unable to setup the local storage pool for Host[-1-Routing]

-- 
====================================================
Natalia Costas Lago
Senior Communications Technician
Galicia Supercomputing Centre (CESGA)
(CESGA on Twitter  |  CESGA on Facebook)

Avenida de Vigo, s/n (Campus Vida)
15705 Santiago de Compostela - SPAIN

E-mail: natalia@cesga.es
Cell: +34 981 56 98 10 (ext. 237)
Fax: +34 981 59 46 16
Web: https://www.cesga.es/
====================================================
[IMPORTANTE] La información contenida en este mensaje y
sus posibles documentos adjuntos es privada y confidencial
y está dirigida únicamente a su destinatario/a. Si usted no
es el/la destinatario/a original de este mensaje, por favor
elimínelo. La distribución o copia de este mensaje no está
autorizada.


Mime
View raw message