cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrija Panic <andrija.pa...@gmail.com>
Subject Re: Unable to add host
Date Wed, 28 Aug 2019 04:31:46 GMT
Is this perhaps a reused KVM host? In that case it's known to fail, you
just need to start the agent servise manually on KVM host, before adding it
again.

Check the /var/log/cloudstack/agent/setup.log for the error messages while
adding/setting up KVM host.

Andrija

On Tue, Aug 27, 2019, 19:43 Nicolas Vazquez <Nicolas.Vazquez@shapeblue.com>
wrote:

> Hi Fariborz,
>
> Which CloudStack version are you using?
> It seems like a connection timeout error, have you verified port 8250 is
> open?
>
>
> 2019-08-27 18:26:07,291 DEBUG [c.c.h.k.d.LibvirtServerDiscoverer]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Timeout, to
> wait for the host connecting to mgt svr, assuming it is failed
> 2019-08-27 18:26:07,293 WARN [c.c.r.ResourceManagerImpl]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Unable to
> find the server resources at http://51.91.247.86<http://51.91.247.86/>
> 2019-08-27 18:26:07,294 INFO [c.c.u.e.CSExceptionErrorCode]
>
> If that is not the case, kindly increase the log level on the agent side
> to DEBUG and paste the output while trying to add the host.
>
> Regards,
> Nicolas Vazquez
>
> ________________________________
> De: Fariborz Navidan <mdvlinquest@gmail.com>
> Enviado: martes, 27 de agosto de 2019 13:30
> Para: users@cloudstack.apache.org
> Asunto: Unable to add host
>
> Hello,
>
> I have setup a new cloudstack management. I am setting up a new advanced
> zone. I am unable to add the kvm host. Bellow is the management server log:
>
> [root@fr-kvm1 agent]# cat ../management/management-server.log
>
> 2019-08-27 18:19:20,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-e9d7f345) (logid:b85ae93e) Begin cleanup
> expired async-jobs
> 2019-08-27 18:19:20,921 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-e9d7f345) (logid:b85ae93e) End cleanup expired
> async-jobs
> 2019-08-27 18:19:26,866 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-aee28ca2) (logid:08603ca9) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:19:30,906 INFO  [c.c.r.ResourceLimitManagerImpl]
> (ResourceCountChecker-1:ctx-f3f085c7) (logid:50310825) Started resource
> counters recalculation periodic task.
> 2019-08-27 18:19:30,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-806d1fc0) (logid:30023a57) Begin cleanup
> expired async-jobs
> 2019-08-27 18:19:30,916 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> (SnapshotPollTask:ctx-8da331fb) (logid:09e138af) Snapshot scheduler.poll is
> being called at 2019-08-27 16:19:30 GMT
> 2019-08-27 18:19:30,917 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> (SnapshotPollTask:ctx-8da331fb) (logid:09e138af) Got 0 snapshots to be
> executed at 2019-08-27 16:19:30 GMT
> 2019-08-27 18:19:30,924 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-806d1fc0) (logid:30023a57) End cleanup expired
> async-jobs
> 2019-08-27 18:19:30,955 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> (ExternalNetworkMonitor-1:ctx-76b32f50) (logid:2469ab38) External devices
> are not used. Skipping external device usage collection
> 2019-08-27 18:19:30,998 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterMonitor-1:ctx-87161167) (logid:93af014a) Found 0 running routers.
> 2019-08-27 18:19:30,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-d4400e54) (logid:f2256a76) Found 0 routers to
> update status.
> 2019-08-27 18:19:31,001 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-d4400e54) (logid:f2256a76) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:19:31,003 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-d4400e54) (logid:f2256a76) Found 0 networks to
> update RvR status.
> 2019-08-27 18:19:40,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-312b8771) (logid:bd56f351) Begin cleanup
> expired async-jobs
> 2019-08-27 18:19:40,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-312b8771) (logid:bd56f351) End cleanup expired
> async-jobs
> 2019-08-27 18:19:46,229 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-56ab68fb) (logid:fc5da2e3) AutoScaling Monitor is
> running...
> 2019-08-27 18:19:46,449 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-5:ctx-20683fd8) (logid:b77cd4e2) HostStatsCollector is
> running...
> 2019-08-27 18:19:46,619 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-ae1c7ccc) (logid:f142989d) StorageCollector is
> running...
> 2019-08-27 18:19:46,630 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-ae1c7ccc) (logid:f142989d) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:19:50,907 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9321ebcc) (logid:0573cf29) Begin cleanup
> expired async-jobs
> 2019-08-27 18:19:50,908 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9321ebcc) (logid:0573cf29) End cleanup expired
> async-jobs
> 2019-08-27 18:19:56,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-ccdf9bc0) (logid:242d9bce) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:20:00,907 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (HostReservationReleaseChecker:ctx-c48b1be7) (logid:43c956f9) Checking if
> any host reservation can be released ...
> 2019-08-27 18:20:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (HostReservationReleaseChecker:ctx-c48b1be7) (logid:43c956f9) Done running
> HostReservationReleaseChecker ...
> 2019-08-27 18:20:00,912 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-c1c58aca) (logid:f751f0de) Begin cleanup
> expired async-jobs
> 2019-08-27 18:20:00,913 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-c1c58aca) (logid:f751f0de) End cleanup expired
> async-jobs
> 2019-08-27 18:20:00,930 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Running Capacity Checker
> ...
> 2019-08-27 18:20:00,930 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) recalculating system
> capacity
> 2019-08-27 18:20:00,930 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Executing cpu/ram capacity
> update
> 2019-08-27 18:20:00,931 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Done executing cpu/ram
> capacity update
> 2019-08-27 18:20:00,931 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Executing storage capacity
> update
> 2019-08-27 18:20:00,932 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Done executing storage
> capacity update
> 2019-08-27 18:20:00,932 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Executing capacity updates
> for public ip and Vlans
> 2019-08-27 18:20:00,937 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Done capacity updates for
> public ip and Vlans
> 2019-08-27 18:20:00,937 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Executing capacity updates
> for private ip
> 2019-08-27 18:20:00,938 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Done executing capacity
> updates for private ip
> 2019-08-27 18:20:00,938 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Done recalculating system
> capacity
> 2019-08-27 18:20:00,945 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-9f2221b5) (logid:4d18443a) Done running Capacity
> Checker ...
> 2019-08-27 18:20:00,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-8b10f027) (logid:14ed07c6) Found 0 routers to
> update status.
> 2019-08-27 18:20:01,001 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-8b10f027) (logid:14ed07c6) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:20:01,003 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-8b10f027) (logid:14ed07c6) Found 0 networks to
> update RvR status.
> 2019-08-27 18:20:05,064 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-293:ctx-e2880880) (logid:26bd3c9f) ===START===
>  137.74.35.102 -- GET  command=listZones&response=json&_=1566920295043
> 2019-08-27 18:20:05,071 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-293:ctx-e2880880 ctx-de45ec7c) (logid:26bd3c9f) CIDRs from
> which account 'Acct[41262157-c8cd-11e9-8ca2-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-27 18:20:05,072 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-321:ctx-a97d6d3f) (logid:c25b9df3) ===START===
>  137.74.35.102 -- GET
>  command=listDomains&response=json&listAll=true&details=min&_=1566920295044
> 2019-08-27 18:20:05,074 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-321:ctx-a97d6d3f ctx-97e84650) (logid:c25b9df3) CIDRs from
> which account 'Acct[41262157-c8cd-11e9-8ca2-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-27 18:20:05,075 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-293:ctx-e2880880 ctx-de45ec7c) (logid:26bd3c9f) ===END===
>  137.74.35.102 -- GET  command=listZones&response=json&_=1566920295043
> 2019-08-27 18:20:05,077 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-321:ctx-a97d6d3f ctx-97e84650) (logid:c25b9df3) ===END===
>  137.74.35.102 -- GET
>  command=listDomains&response=json&listAll=true&details=min&_=1566920295044
> 2019-08-27 18:20:05,080 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-311:ctx-f557914c) (logid:e0f1a549) ===START===
>  137.74.35.102 -- GET  command=listHostTags&response=json&_=1566920295045
> 2019-08-27 18:20:05,082 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-311:ctx-f557914c ctx-becaa306) (logid:e0f1a549) CIDRs from
> which account 'Acct[41262157-c8cd-11e9-8ca2-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-27 18:20:05,084 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-311:ctx-f557914c ctx-becaa306) (logid:e0f1a549) ===END===
>  137.74.35.102 -- GET  command=listHostTags&response=json&_=1566920295045
> 2019-08-27 18:20:05,204 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-321:ctx-38c4ff38) (logid:1f63d2cf) ===START===
>  137.74.35.102 -- GET
>
>  command=listPods&zoneid=0182ff76-fd63-45d2-9414-eeacc1360420&response=json&_=1566920295046
> 2019-08-27 18:20:05,212 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-321:ctx-38c4ff38 ctx-2d3520b2) (logid:1f63d2cf) CIDRs from
> which account 'Acct[41262157-c8cd-11e9-8ca2-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-27 18:20:05,227 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-321:ctx-38c4ff38 ctx-2d3520b2) (logid:1f63d2cf) ===END===
>  137.74.35.102 -- GET
>
>  command=listPods&zoneid=0182ff76-fd63-45d2-9414-eeacc1360420&response=json&_=1566920295046
> 2019-08-27 18:20:05,341 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-311:ctx-30d72e93) (logid:c0d825b6) ===START===
>  137.74.35.102 -- GET
>
>  command=listClusters&podid=df295e72-c627-47f0-8c84-4c51a295b305&response=json&_=1566920295047
> 2019-08-27 18:20:05,349 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-311:ctx-30d72e93 ctx-06d01796) (logid:c0d825b6) CIDRs from
> which account 'Acct[41262157-c8cd-11e9-8ca2-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-27 18:20:05,365 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-311:ctx-30d72e93 ctx-06d01796) (logid:c0d825b6) ===END===
>  137.74.35.102 -- GET
>
>  command=listClusters&podid=df295e72-c627-47f0-8c84-4c51a295b305&response=json&_=1566920295047
> 2019-08-27 18:20:10,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-19507ba3) (logid:6391dc13) Begin cleanup
> expired async-jobs
> 2019-08-27 18:20:10,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-19507ba3) (logid:6391dc13) End cleanup expired
> async-jobs
> 2019-08-27 18:20:20,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-4ae37d20) (logid:8201e3ec) Begin cleanup
> expired async-jobs
> 2019-08-27 18:20:20,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-4ae37d20) (logid:8201e3ec) End cleanup expired
> async-jobs
> 2019-08-27 18:20:26,866 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-3c880805) (logid:0bc5f74b) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:20:30,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d579592d) (logid:bf4907ae) Begin cleanup
> expired async-jobs
> 2019-08-27 18:20:30,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d579592d) (logid:bf4907ae) End cleanup expired
> async-jobs
> 2019-08-27 18:20:31,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-f325a584) (logid:c69f667d) Found 0 routers to
> update status.
> 2019-08-27 18:20:31,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-f325a584) (logid:c69f667d) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:20:31,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-f325a584) (logid:c69f667d) Found 0 networks to
> update RvR status.
> 2019-08-27 18:20:31,098 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-321:ctx-4b9d5ee5) (logid:1c7f0e2c) ===START===
>  137.74.35.102 -- POST  command=addHost&response=json
> 2019-08-27 18:20:31,106 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) CIDRs from
> which account 'Acct[41262157-c8cd-11e9-8ca2-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-27 18:20:31,117 WARN  [c.c.a.d.ParamGenericValidationWorker]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Received
> unknown parameters for command addHost. Unknown parameters : clustertype
> 2019-08-27 18:20:31,123 INFO  [c.c.r.ResourceManagerImpl]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Trying to
> add a new host at http://51.91.247.86 in data center 1
> 2019-08-27 18:20:31,223 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Executing
> cmd: lsmod|grep kvm
> 2019-08-27 18:20:32,286 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) SSH command:
> lsmod|grep kvm
> SSH command output:kvm_intel             188683  0
> kvm                   621392  1 kvm_intel
> irqbypass              13503  1 kvm
>
>
> 2019-08-27 18:20:32,289 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Executing
> cmd: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup
> /etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
> 2019-08-27 18:20:33,776 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) SSH command:
> sudo /usr/share/cloudstack-common/scripts/util/keystore-setup
> /etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
> SSH command output:
>
> 2019-08-27 18:20:33,792 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Executing
> cmd: sudo /usr/share/cloudstack-common/scripts/util/keystore-cert-import
> /etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
> 2019-08-27 18:20:35,536 DEBUG [c.c.h.k.d.LibvirtServerDiscoverer]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Succeeded to
> import certificate in the keystore for agent on the KVM host: 51.91.247.86.
> Agent secured and trusted.
> 2019-08-27 18:20:35,537 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Executing
> cmd: cloudstack-setup-agent  -m 178.33.230.41 -z 1 -p 1 -c 1 -g
> bd6dea31-32c6-37de-9b69-48add0e8f8e0 -a -s  --pubNic=cloudbr0
> --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm
> 2019-08-27 18:20:36,025 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-f4c9abe8) (logid:9ae08218) Resetting hosts suitable for reconnect
> 2019-08-27 18:20:36,027 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-f4c9abe8) (logid:9ae08218) Completed resetting hosts suitable for
> reconnect
> 2019-08-27 18:20:36,027 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-f4c9abe8) (logid:9ae08218) Acquiring hosts for clusters already
> owned by this management server
> 2019-08-27 18:20:36,028 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-f4c9abe8) (logid:9ae08218) Completed acquiring hosts for clusters
> already owned by this management server
> 2019-08-27 18:20:36,028 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-f4c9abe8) (logid:9ae08218) Acquiring hosts for clusters not owned
> by any management server
> 2019-08-27 18:20:36,029 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-f4c9abe8) (logid:9ae08218) Completed acquiring hosts for clusters
> not owned by any management server
> 2019-08-27 18:20:40,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-40798b95) (logid:27e5b622) Begin cleanup
> expired async-jobs
> 2019-08-27 18:20:40,924 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-40798b95) (logid:27e5b622) End cleanup expired
> async-jobs
> 2019-08-27 18:20:46,231 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-eeea8ba1) (logid:9553fb8b) AutoScaling Monitor is
> running...
> 2019-08-27 18:20:46,453 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-e8ed0ab0) (logid:7e80b3c3) HostStatsCollector is
> running...
> 2019-08-27 18:20:46,632 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-b2d2078e) (logid:5a8e9f2e) StorageCollector is
> running...
> 2019-08-27 18:20:46,643 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-b2d2078e) (logid:5a8e9f2e) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:20:50,916 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-039d054d) (logid:84cedcc6) Begin cleanup
> expired async-jobs
> 2019-08-27 18:20:50,924 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-039d054d) (logid:84cedcc6) End cleanup expired
> async-jobs
> 2019-08-27 18:20:56,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-69ba7348) (logid:03db2173) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:21:00,906 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-429a46ca) (logid:6e72f6ae) Begin cleanup
> expired async-jobs
> 2019-08-27 18:21:00,907 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-429a46ca) (logid:6e72f6ae) End cleanup expired
> async-jobs
> 2019-08-27 18:21:00,998 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-33c58636) (logid:690fedd2) Found 0 routers to
> update status.
> 2019-08-27 18:21:00,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-33c58636) (logid:690fedd2) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:21:00,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-33c58636) (logid:690fedd2) Found 0 networks to
> update RvR status.
> 2019-08-27 18:21:07,273 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) SSH command:
> cloudstack-setup-agent  -m 178.33.230.41 -z 1 -p 1 -c 1 -g
> bd6dea31-32c6-37de-9b69-48add0e8f8e0 -a -s  --pubNic=cloudbr0
> --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm
> SSH command output:Starting to configure your system:
> Configure SElinux ...         [OK]
> Configure Network ...         [OK]
> Configure Libvirt ...         [OK]
> Configure Firewall ...        [OK]
> Configure Nfs ...             [OK]
> Configure cloudAgent ...      [OK]
> CloudStack Agent setup is done!
>
>
> 2019-08-27 18:21:10,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-24203593) (logid:8f77d5c3) Begin cleanup
> expired async-jobs
> 2019-08-27 18:21:10,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-24203593) (logid:8f77d5c3) End cleanup expired
> async-jobs
> 2019-08-27 18:21:20,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-c1610f34) (logid:cfb35e59) Begin cleanup
> expired async-jobs
> 2019-08-27 18:21:20,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-c1610f34) (logid:cfb35e59) End cleanup expired
> async-jobs
> 2019-08-27 18:21:26,871 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-a01c28d8) (logid:dc18e099) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:21:30,906 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-301274f1) (logid:20a91ec9) Begin cleanup
> expired async-jobs
> 2019-08-27 18:21:30,907 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-301274f1) (logid:20a91ec9) End cleanup expired
> async-jobs
> 2019-08-27 18:21:30,998 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-7470fa3b) (logid:5a58570f) Found 0 routers to
> update status.
> 2019-08-27 18:21:30,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-7470fa3b) (logid:5a58570f) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:21:30,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-7470fa3b) (logid:5a58570f) Found 0 networks to
> update RvR status.
> 2019-08-27 18:21:40,912 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-3e204124) (logid:8675566f) Begin cleanup
> expired async-jobs
> 2019-08-27 18:21:40,913 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-3e204124) (logid:8675566f) End cleanup expired
> async-jobs
> 2019-08-27 18:21:46,234 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-5:ctx-b4934eac) (logid:14e0c459) AutoScaling Monitor is
> running...
> 2019-08-27 18:21:46,458 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-03f971f8) (logid:c7517e7d) HostStatsCollector is
> running...
> 2019-08-27 18:21:46,644 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-cf5a6e3f) (logid:50cc6740) StorageCollector is
> running...
> 2019-08-27 18:21:46,655 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-cf5a6e3f) (logid:50cc6740) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:21:50,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-895dfc29) (logid:a6b5f784) Begin cleanup
> expired async-jobs
> 2019-08-27 18:21:50,924 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-895dfc29) (logid:a6b5f784) End cleanup expired
> async-jobs
> 2019-08-27 18:21:56,866 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-789ef6b4) (logid:df80d423) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:22:00,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2961bb1d) (logid:18bdadc2) Begin cleanup
> expired async-jobs
> 2019-08-27 18:22:00,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2961bb1d) (logid:18bdadc2) End cleanup expired
> async-jobs
> 2019-08-27 18:22:01,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-d2b43c06) (logid:991d70f0) Found 0 routers to
> update status.
> 2019-08-27 18:22:01,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-d2b43c06) (logid:991d70f0) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:22:01,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-d2b43c06) (logid:991d70f0) Found 0 networks to
> update RvR status.
> 2019-08-27 18:22:06,022 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-b2528429) (logid:a11d6bff) Resetting hosts suitable for reconnect
> 2019-08-27 18:22:06,022 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-b2528429) (logid:a11d6bff) Completed resetting hosts suitable for
> reconnect
> 2019-08-27 18:22:06,022 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-b2528429) (logid:a11d6bff) Acquiring hosts for clusters already
> owned by this management server
> 2019-08-27 18:22:06,022 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-b2528429) (logid:a11d6bff) Completed acquiring hosts for clusters
> already owned by this management server
> 2019-08-27 18:22:06,022 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-b2528429) (logid:a11d6bff) Acquiring hosts for clusters not owned
> by any management server
> 2019-08-27 18:22:06,022 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-b2528429) (logid:a11d6bff) Completed acquiring hosts for clusters
> not owned by any management server
> 2019-08-27 18:22:10,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-dd284312) (logid:6e3dd166) Begin cleanup
> expired async-jobs
> 2019-08-27 18:22:10,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-dd284312) (logid:6e3dd166) End cleanup expired
> async-jobs
> 2019-08-27 18:22:20,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d9a2ff6a) (logid:cf5777d5) Begin cleanup
> expired async-jobs
> 2019-08-27 18:22:20,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d9a2ff6a) (logid:cf5777d5) End cleanup expired
> async-jobs
> 2019-08-27 18:22:26,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-07411a3e) (logid:effcf164) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:22:30,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9074c118) (logid:1713469d) Begin cleanup
> expired async-jobs
> 2019-08-27 18:22:30,921 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9074c118) (logid:1713469d) End cleanup expired
> async-jobs
> 2019-08-27 18:22:31,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-e4a9605a) (logid:54e22a2d) Found 0 routers to
> update status.
> 2019-08-27 18:22:31,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-e4a9605a) (logid:54e22a2d) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:22:31,003 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-e4a9605a) (logid:54e22a2d) Found 0 networks to
> update RvR status.
> 2019-08-27 18:22:40,907 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ac93147e) (logid:1255738c) Begin cleanup
> expired async-jobs
> 2019-08-27 18:22:40,908 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ac93147e) (logid:1255738c) End cleanup expired
> async-jobs
> 2019-08-27 18:22:46,236 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-eaa9939a) (logid:1a5882e3) AutoScaling Monitor is
> running...
> 2019-08-27 18:22:46,463 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-5:ctx-60bb01b9) (logid:2f64b5bb) HostStatsCollector is
> running...
> 2019-08-27 18:22:46,656 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-df2298f5) (logid:13a6471a) StorageCollector is
> running...
> 2019-08-27 18:22:46,667 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-df2298f5) (logid:13a6471a) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:22:50,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ab806e34) (logid:27d9e81b) Begin cleanup
> expired async-jobs
> 2019-08-27 18:22:50,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ab806e34) (logid:27d9e81b) End cleanup expired
> async-jobs
> 2019-08-27 18:22:56,868 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-0ff0b241) (logid:4497cfff) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:23:00,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-001a6de2) (logid:88c6b27e) Begin cleanup
> expired async-jobs
> 2019-08-27 18:23:00,916 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-001a6de2) (logid:88c6b27e) End cleanup expired
> async-jobs
> 2019-08-27 18:23:00,998 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-d9901de1) (logid:a70c4b1f) Found 0 routers to
> update status.
> 2019-08-27 18:23:00,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-d9901de1) (logid:a70c4b1f) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:23:00,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-d9901de1) (logid:a70c4b1f) Found 0 networks to
> update RvR status.
> 2019-08-27 18:23:10,906 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2d786385) (logid:e8c477c4) Begin cleanup
> expired async-jobs
> 2019-08-27 18:23:10,907 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2d786385) (logid:e8c477c4) End cleanup expired
> async-jobs
> 2019-08-27 18:23:20,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-3f7feeac) (logid:4d309948) Begin cleanup
> expired async-jobs
> 2019-08-27 18:23:20,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-3f7feeac) (logid:4d309948) End cleanup expired
> async-jobs
> 2019-08-27 18:23:26,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-acee68c7) (logid:635541b1) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:23:30,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7cd8319b) (logid:3bb850c3) Begin cleanup
> expired async-jobs
> 2019-08-27 18:23:30,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7cd8319b) (logid:3bb850c3) End cleanup expired
> async-jobs
> 2019-08-27 18:23:31,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-517a2425) (logid:078776e0) Found 0 routers to
> update status.
> 2019-08-27 18:23:31,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-517a2425) (logid:078776e0) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:23:31,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-517a2425) (logid:078776e0) Found 0 networks to
> update RvR status.
> 2019-08-27 18:23:36,025 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-322679ac) (logid:ff55f033) Resetting hosts suitable for reconnect
> 2019-08-27 18:23:36,027 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-322679ac) (logid:ff55f033) Completed resetting hosts suitable for
> reconnect
> 2019-08-27 18:23:36,028 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-322679ac) (logid:ff55f033) Acquiring hosts for clusters already
> owned by this management server
> 2019-08-27 18:23:36,029 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-322679ac) (logid:ff55f033) Completed acquiring hosts for clusters
> already owned by this management server
> 2019-08-27 18:23:36,029 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-322679ac) (logid:ff55f033) Acquiring hosts for clusters not owned
> by any management server
> 2019-08-27 18:23:36,030 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-322679ac) (logid:ff55f033) Completed acquiring hosts for clusters
> not owned by any management server
> 2019-08-27 18:23:40,906 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7c03350f) (logid:5fae63eb) Begin cleanup
> expired async-jobs
> 2019-08-27 18:23:40,908 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7c03350f) (logid:5fae63eb) End cleanup expired
> async-jobs
> 2019-08-27 18:23:46,239 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-b95ffc6b) (logid:79e63708) AutoScaling Monitor is
> running...
> 2019-08-27 18:23:46,468 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-bf184091) (logid:a52a4e48) HostStatsCollector is
> running...
> 2019-08-27 18:23:46,669 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-49aa97a2) (logid:1cfe88d2) StorageCollector is
> running...
> 2019-08-27 18:23:46,680 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-49aa97a2) (logid:1cfe88d2) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:23:50,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ac2f46da) (logid:94191348) Begin cleanup
> expired async-jobs
> 2019-08-27 18:23:50,924 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ac2f46da) (logid:94191348) End cleanup expired
> async-jobs
> 2019-08-27 18:23:56,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-e07860d7) (logid:48874217) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:24:00,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9877914d) (logid:129f4263) Begin cleanup
> expired async-jobs
> 2019-08-27 18:24:00,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9877914d) (logid:129f4263) End cleanup expired
> async-jobs
> 2019-08-27 18:24:01,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-fb323798) (logid:02c0d0ce) Found 0 routers to
> update status.
> 2019-08-27 18:24:01,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-fb323798) (logid:02c0d0ce) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:24:01,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-fb323798) (logid:02c0d0ce) Found 0 networks to
> update RvR status.
> 2019-08-27 18:24:10,913 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-bd488ee9) (logid:b9866bde) Begin cleanup
> expired async-jobs
> 2019-08-27 18:24:10,921 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-bd488ee9) (logid:b9866bde) End cleanup expired
> async-jobs
> 2019-08-27 18:24:20,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-b7f8ce9b) (logid:420a3696) Begin cleanup
> expired async-jobs
> 2019-08-27 18:24:20,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-b7f8ce9b) (logid:420a3696) End cleanup expired
> async-jobs
> 2019-08-27 18:24:26,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-4cd1a6c4) (logid:4db4d10c) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:24:30,906 INFO  [c.c.r.ResourceLimitManagerImpl]
> (ResourceCountChecker-1:ctx-e3ba647f) (logid:2993b077) Started resource
> counters recalculation periodic task.
> 2019-08-27 18:24:30,912 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-1472096a) (logid:c451e053) Begin cleanup
> expired async-jobs
> 2019-08-27 18:24:30,913 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> (SnapshotPollTask:ctx-4a613430) (logid:9a5c7d9f) Snapshot scheduler.poll is
> being called at 2019-08-27 16:24:30 GMT
> 2019-08-27 18:24:30,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-1472096a) (logid:c451e053) End cleanup expired
> async-jobs
> 2019-08-27 18:24:30,916 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> (SnapshotPollTask:ctx-4a613430) (logid:9a5c7d9f) Got 0 snapshots to be
> executed at 2019-08-27 16:24:30 GMT
> 2019-08-27 18:24:30,952 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> (ExternalNetworkMonitor-1:ctx-fac2ea3b) (logid:a4708073) External devices
> are not used. Skipping external device usage collection
> 2019-08-27 18:24:30,979 INFO  [o.a.c.e.o.NetworkOrchestrator]
> (Network-Scavenger-1:ctx-9ab61b45) (logid:7e2c6051) NetworkGarbageCollector
> uses '600' seconds for GC interval.
> 2019-08-27 18:24:30,987 INFO  [c.c.h.HighAvailabilityManagerImpl]
> (HA-2:ctx-37c12e9b) (logid:06629de8) checking health of usage server
> 2019-08-27 18:24:30,990 DEBUG [c.c.h.HighAvailabilityManagerImpl]
> (HA-2:ctx-37c12e9b) (logid:06629de8) usage server running? false,
> heartbeat: null
> 2019-08-27 18:24:30,992 WARN  [c.c.a.AlertManagerImpl] (HA-2:ctx-37c12e9b)
> (logid:06629de8) AlertType:: 13 | dataCenterId:: 0 | podId:: 0 |
> clusterId:: null | message:: No usage server process running
> 2019-08-27 18:24:30,995 DEBUG [c.c.a.AlertManagerImpl] (HA-2:ctx-37c12e9b)
> (logid:06629de8) Have already sent: 1 emails for alert type '13' --
> skipping send email
> 2019-08-27 18:24:30,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterMonitor-1:ctx-be50d5f2) (logid:aa42fed6) Found 0 running routers.
> 2019-08-27 18:24:30,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bd4f4e99) (logid:b8fd5bc4) Found 0 routers to
> update status.
> 2019-08-27 18:24:31,001 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bd4f4e99) (logid:b8fd5bc4) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:24:31,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bd4f4e99) (logid:b8fd5bc4) Found 0 networks to
> update RvR status.
> 2019-08-27 18:24:40,906 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5e6b69b5) (logid:ec7bb2ef) Begin cleanup
> expired async-jobs
> 2019-08-27 18:24:40,907 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5e6b69b5) (logid:ec7bb2ef) End cleanup expired
> async-jobs
> 2019-08-27 18:24:40,925 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> (LBHealthCheck-1:ctx-d38d22b1) (logid:ad33afe2) LB HealthCheck Manager is
> running and getting the updates from LB providers and updating service
> status
> 2019-08-27 18:24:40,927 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> (LBHealthCheck-1:ctx-d38d22b1) (logid:ad33afe2) LB HealthCheck Manager is
> running and getting the updates from LB providers and updating service
> status
> 2019-08-27 18:24:46,241 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-d45bdc40) (logid:a70b2ecc) AutoScaling Monitor is
> running...
> 2019-08-27 18:24:46,472 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-ac4cf06f) (logid:90e99b27) HostStatsCollector is
> running...
> 2019-08-27 18:24:46,681 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-9e304fe7) (logid:481e3b8b) StorageCollector is
> running...
> 2019-08-27 18:24:46,692 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-9e304fe7) (logid:481e3b8b) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:24:50,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-cffc9982) (logid:ec1b6bd4) Begin cleanup
> expired async-jobs
> 2019-08-27 18:24:50,924 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-cffc9982) (logid:ec1b6bd4) End cleanup expired
> async-jobs
> 2019-08-27 18:24:56,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-4c4422fc) (logid:94667a55) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:25:00,908 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (HostReservationReleaseChecker:ctx-62d605e9) (logid:31b54e62) Checking if
> any host reservation can be released ...
> 2019-08-27 18:25:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (HostReservationReleaseChecker:ctx-62d605e9) (logid:31b54e62) Done running
> HostReservationReleaseChecker ...
> 2019-08-27 18:25:00,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5a0cdb59) (logid:438e6cbe) Begin cleanup
> expired async-jobs
> 2019-08-27 18:25:00,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5a0cdb59) (logid:438e6cbe) End cleanup expired
> async-jobs
> 2019-08-27 18:25:00,930 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Running Capacity Checker
> ...
> 2019-08-27 18:25:00,930 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) recalculating system
> capacity
> 2019-08-27 18:25:00,930 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Executing cpu/ram capacity
> update
> 2019-08-27 18:25:00,932 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Done executing cpu/ram
> capacity update
> 2019-08-27 18:25:00,932 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Executing storage capacity
> update
> 2019-08-27 18:25:00,933 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Done executing storage
> capacity update
> 2019-08-27 18:25:00,933 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Executing capacity updates
> for public ip and Vlans
> 2019-08-27 18:25:00,942 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Done capacity updates for
> public ip and Vlans
> 2019-08-27 18:25:00,942 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Executing capacity updates
> for private ip
> 2019-08-27 18:25:00,943 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Done executing capacity
> updates for private ip
> 2019-08-27 18:25:00,943 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Done recalculating system
> capacity
> 2019-08-27 18:25:00,949 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-7d4cfe66) (logid:eb88a941) Done running Capacity
> Checker ...
> 2019-08-27 18:25:00,999 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-a6191fba) (logid:88766c41) Found 0 routers to
> update status.
> 2019-08-27 18:25:01,001 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-a6191fba) (logid:88766c41) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:25:01,003 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-a6191fba) (logid:88766c41) Found 0 networks to
> update RvR status.
> 2019-08-27 18:25:06,026 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8f861e34) (logid:8d6db5ee) Resetting hosts suitable for reconnect
> 2019-08-27 18:25:06,028 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8f861e34) (logid:8d6db5ee) Completed resetting hosts suitable for
> reconnect
> 2019-08-27 18:25:06,028 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8f861e34) (logid:8d6db5ee) Acquiring hosts for clusters already
> owned by this management server
> 2019-08-27 18:25:06,029 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8f861e34) (logid:8d6db5ee) Completed acquiring hosts for clusters
> already owned by this management server
> 2019-08-27 18:25:06,029 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8f861e34) (logid:8d6db5ee) Acquiring hosts for clusters not owned
> by any management server
> 2019-08-27 18:25:06,030 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8f861e34) (logid:8d6db5ee) Completed acquiring hosts for clusters
> not owned by any management server
> 2019-08-27 18:25:10,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-8ab5a69a) (logid:ea7d6eb0) Begin cleanup
> expired async-jobs
> 2019-08-27 18:25:10,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-8ab5a69a) (logid:ea7d6eb0) End cleanup expired
> async-jobs
> 2019-08-27 18:25:20,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-06c6b6ac) (logid:7b30442e) Begin cleanup
> expired async-jobs
> 2019-08-27 18:25:20,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-06c6b6ac) (logid:7b30442e) End cleanup expired
> async-jobs
> 2019-08-27 18:25:26,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-8a0cab4b) (logid:d043bcc5) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:25:30,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-70305999) (logid:25acf29f) Begin cleanup
> expired async-jobs
> 2019-08-27 18:25:30,924 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-70305999) (logid:25acf29f) End cleanup expired
> async-jobs
> 2019-08-27 18:25:31,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-a3c8f201) (logid:7c4e892a) Found 0 routers to
> update status.
> 2019-08-27 18:25:31,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-a3c8f201) (logid:7c4e892a) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:25:31,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-a3c8f201) (logid:7c4e892a) Found 0 networks to
> update RvR status.
> 2019-08-27 18:25:40,906 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-38fdd117) (logid:26490bef) Begin cleanup
> expired async-jobs
> 2019-08-27 18:25:40,908 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-38fdd117) (logid:26490bef) End cleanup expired
> async-jobs
> 2019-08-27 18:25:46,244 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-f0911e57) (logid:29535f2f) AutoScaling Monitor is
> running...
> 2019-08-27 18:25:46,477 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-3dccee41) (logid:2fa4c3d5) HostStatsCollector is
> running...
> 2019-08-27 18:25:46,694 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-10e227a0) (logid:13fe03f7) StorageCollector is
> running...
> 2019-08-27 18:25:46,705 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-10e227a0) (logid:13fe03f7) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:25:50,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-05c40c76) (logid:e5b21312) Begin cleanup
> expired async-jobs
> 2019-08-27 18:25:50,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-05c40c76) (logid:e5b21312) End cleanup expired
> async-jobs
> 2019-08-27 18:25:56,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-1f22fe2a) (logid:e9ac74a3) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:26:00,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-4f688a34) (logid:2c01616e) Begin cleanup
> expired async-jobs
> 2019-08-27 18:26:00,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-4f688a34) (logid:2c01616e) End cleanup expired
> async-jobs
> 2019-08-27 18:26:01,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-116a7622) (logid:75edfaeb) Found 0 routers to
> update status.
> 2019-08-27 18:26:01,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-116a7622) (logid:75edfaeb) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:26:01,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-116a7622) (logid:75edfaeb) Found 0 networks to
> update RvR status.
> 2019-08-27 18:26:07,291 DEBUG [c.c.h.k.d.LibvirtServerDiscoverer]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Timeout, to
> wait for the host connecting to mgt svr, assuming it is failed
> 2019-08-27 18:26:07,293 WARN  [c.c.r.ResourceManagerImpl]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Unable to
> find the server resources at http://51.91.247.86
> 2019-08-27 18:26:07,294 INFO  [c.c.u.e.CSExceptionErrorCode]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Could not
> find exception: com.cloud.exception.DiscoveryException in error code list
> for exceptions
> 2019-08-27 18:26:07,295 WARN  [o.a.c.a.c.a.h.AddHostCmd]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Exception:
> com.cloud.exception.DiscoveryException: Unable to add the host
>         at
>
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:824)
>         at
>
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:610)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
>
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
>         at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
>         at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>         at
>
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
>         at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
>         at
>
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
>         at com.sun.proxy.$Proxy174.discoverHosts(Unknown Source)
>         at
>
> org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:142)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
>         at com.cloud.api.ApiServer.queueCommand(ApiServer.java:758)
>         at com.cloud.api.ApiServer.handleRequest(ApiServer.java:582)
>         at
> com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:310)
>         at com.cloud.api.ApiServlet$1.run(ApiServlet.java:130)
>         at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:127)
>         at com.cloud.api.ApiServlet.doPost(ApiServlet.java:94)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:706)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:791)
>         at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:852)
>         at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
>         at
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>         at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>         at
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>         at
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
>         at
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
>         at
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
>         at
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
>         at
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
>         at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
>         at
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
>         at
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
>         at
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
>         at
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>         at
>
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
>         at
>
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
>         at
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>         at org.eclipse.jetty.server.Server.handle(Server.java:530)
>         at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
>         at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
>         at
> org.eclipse.jetty.io
> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
>         at org.eclipse.jetty.io
> .FillInterest.fillable(FillInterest.java:102)
>         at
> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
>         at
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
>         at
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
>         at
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
>         at
>
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
>         at
>
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
>         at
>
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
>         at java.lang.Thread.run(Thread.java:748)
> 2019-08-27 18:26:07,299 INFO  [c.c.a.ApiServer]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) Unable to
> add the host
> 2019-08-27 18:26:07,300 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-321:ctx-4b9d5ee5 ctx-61f92e0c) (logid:1c7f0e2c) ===END===
>  137.74.35.102 -- POST  command=addHost&response=json
> 2019-08-27 18:26:10,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-86d981e4) (logid:3d6370bf) Begin cleanup
> expired async-jobs
> 2019-08-27 18:26:10,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-86d981e4) (logid:3d6370bf) End cleanup expired
> async-jobs
> 2019-08-27 18:26:20,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d03e1ca9) (logid:515d9f8d) Begin cleanup
> expired async-jobs
> 2019-08-27 18:26:20,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d03e1ca9) (logid:515d9f8d) End cleanup expired
> async-jobs
> 2019-08-27 18:26:26,866 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-8baed9f4) (logid:91787797) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:26:30,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2b046fba) (logid:b60d8bb5) Begin cleanup
> expired async-jobs
> 2019-08-27 18:26:30,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2b046fba) (logid:b60d8bb5) End cleanup expired
> async-jobs
> 2019-08-27 18:26:31,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bbe713f0) (logid:d667ccdf) Found 0 routers to
> update status.
> 2019-08-27 18:26:31,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bbe713f0) (logid:d667ccdf) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:26:31,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bbe713f0) (logid:d667ccdf) Found 0 networks to
> update RvR status.
> 2019-08-27 18:26:36,026 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-02626419) (logid:80903e65) Resetting hosts suitable for reconnect
> 2019-08-27 18:26:36,028 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-02626419) (logid:80903e65) Completed resetting hosts suitable for
> reconnect
> 2019-08-27 18:26:36,028 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-02626419) (logid:80903e65) Acquiring hosts for clusters already
> owned by this management server
> 2019-08-27 18:26:36,029 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-02626419) (logid:80903e65) Completed acquiring hosts for clusters
> already owned by this management server
> 2019-08-27 18:26:36,029 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-02626419) (logid:80903e65) Acquiring hosts for clusters not owned
> by any management server
> 2019-08-27 18:26:36,030 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-02626419) (logid:80903e65) Completed acquiring hosts for clusters
> not owned by any management server
> 2019-08-27 18:26:40,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d3882961) (logid:bb6d045f) Begin cleanup
> expired async-jobs
> 2019-08-27 18:26:40,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d3882961) (logid:bb6d045f) End cleanup expired
> async-jobs
> 2019-08-27 18:26:46,247 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-4f0f2e4e) (logid:a4fd473a) AutoScaling Monitor is
> running...
> 2019-08-27 18:26:46,482 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-a7a1d906) (logid:1d0c3400) HostStatsCollector is
> running...
> 2019-08-27 18:26:46,706 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-5:ctx-0a30b02a) (logid:725df9e4) StorageCollector is
> running...
> 2019-08-27 18:26:46,717 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-5:ctx-0a30b02a) (logid:725df9e4) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:26:50,907 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ea102b2b) (logid:37d32c1a) Begin cleanup
> expired async-jobs
> 2019-08-27 18:26:50,908 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ea102b2b) (logid:37d32c1a) End cleanup expired
> async-jobs
> 2019-08-27 18:26:56,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-7fc1d2a9) (logid:1d951ec9) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:27:00,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-a6ec77da) (logid:41f0ebc3) Begin cleanup
> expired async-jobs
> 2019-08-27 18:27:00,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-a6ec77da) (logid:41f0ebc3) End cleanup expired
> async-jobs
> 2019-08-27 18:27:01,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-13147470) (logid:5b192aec) Found 0 routers to
> update status.
> 2019-08-27 18:27:01,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-13147470) (logid:5b192aec) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:27:01,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-13147470) (logid:5b192aec) Found 0 networks to
> update RvR status.
> 2019-08-27 18:27:10,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-58f559d2) (logid:ae05a3a6) Begin cleanup
> expired async-jobs
> 2019-08-27 18:27:10,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-58f559d2) (logid:ae05a3a6) End cleanup expired
> async-jobs
> 2019-08-27 18:27:20,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-52e7dce3) (logid:540a1e75) Begin cleanup
> expired async-jobs
> 2019-08-27 18:27:20,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-52e7dce3) (logid:540a1e75) End cleanup expired
> async-jobs
> 2019-08-27 18:27:26,866 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-cfcccd6c) (logid:f9587a5d) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:27:30,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ac186bb4) (logid:2f527051) Begin cleanup
> expired async-jobs
> 2019-08-27 18:27:30,922 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ac186bb4) (logid:2f527051) End cleanup expired
> async-jobs
> 2019-08-27 18:27:31,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-08a0f60e) (logid:853c9528) Found 0 routers to
> update status.
> 2019-08-27 18:27:31,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-08a0f60e) (logid:853c9528) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:27:31,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-08a0f60e) (logid:853c9528) Found 0 networks to
> update RvR status.
> 2019-08-27 18:27:40,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9ba10e03) (logid:52cc9f52) Begin cleanup
> expired async-jobs
> 2019-08-27 18:27:40,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9ba10e03) (logid:52cc9f52) End cleanup expired
> async-jobs
> 2019-08-27 18:27:46,249 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-5dc93932) (logid:ec944b36) AutoScaling Monitor is
> running...
> 2019-08-27 18:27:46,486 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-d21f6e46) (logid:3904136a) HostStatsCollector is
> running...
> 2019-08-27 18:27:46,718 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-3c75c48d) (logid:2809635f) StorageCollector is
> running...
> 2019-08-27 18:27:46,729 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-3c75c48d) (logid:2809635f) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:27:50,906 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7d48df8f) (logid:0402444c) Begin cleanup
> expired async-jobs
> 2019-08-27 18:27:50,908 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7d48df8f) (logid:0402444c) End cleanup expired
> async-jobs
> 2019-08-27 18:27:56,872 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-daa40045) (logid:66ae8a04) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:28:00,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-869636e6) (logid:a2b34b8c) Begin cleanup
> expired async-jobs
> 2019-08-27 18:28:00,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-869636e6) (logid:a2b34b8c) End cleanup expired
> async-jobs
> 2019-08-27 18:28:01,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-9aa764a6) (logid:9011a671) Found 0 routers to
> update status.
> 2019-08-27 18:28:01,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-9aa764a6) (logid:9011a671) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:28:01,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-9aa764a6) (logid:9011a671) Found 0 networks to
> update RvR status.
> 2019-08-27 18:28:06,026 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7eba2d38) (logid:19f5737f) Resetting hosts suitable for reconnect
> 2019-08-27 18:28:06,028 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7eba2d38) (logid:19f5737f) Completed resetting hosts suitable for
> reconnect
> 2019-08-27 18:28:06,028 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7eba2d38) (logid:19f5737f) Acquiring hosts for clusters already
> owned by this management server
> 2019-08-27 18:28:06,029 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7eba2d38) (logid:19f5737f) Completed acquiring hosts for clusters
> already owned by this management server
> 2019-08-27 18:28:06,029 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7eba2d38) (logid:19f5737f) Acquiring hosts for clusters not owned
> by any management server
> 2019-08-27 18:28:06,030 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7eba2d38) (logid:19f5737f) Completed acquiring hosts for clusters
> not owned by any management server
> 2019-08-27 18:28:10,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-59ebd633) (logid:5e727169) Begin cleanup
> expired async-jobs
> 2019-08-27 18:28:10,924 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-59ebd633) (logid:5e727169) End cleanup expired
> async-jobs
> 2019-08-27 18:28:20,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d36d2f64) (logid:33389890) Begin cleanup
> expired async-jobs
> 2019-08-27 18:28:20,924 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d36d2f64) (logid:33389890) End cleanup expired
> async-jobs
> 2019-08-27 18:28:26,871 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-01f60c08) (logid:3412f0a3) Skip capacity scan as there
> is no Primary Storage in 'Up' state
> 2019-08-27 18:28:30,914 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9c182c3d) (logid:ad79b225) Begin cleanup
> expired async-jobs
> 2019-08-27 18:28:30,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9c182c3d) (logid:ad79b225) End cleanup expired
> async-jobs
> 2019-08-27 18:28:31,000 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-0762fd63) (logid:6609bdd3) Found 0 routers to
> update status.
> 2019-08-27 18:28:31,002 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-0762fd63) (logid:6609bdd3) Found 0 VPC networks
> to update Redundant State.
> 2019-08-27 18:28:31,004 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-0762fd63) (logid:6609bdd3) Found 0 networks to
> update RvR status.
> 2019-08-27 18:28:40,915 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-cdd088b6) (logid:81c638a6) Begin cleanup
> expired async-jobs
> 2019-08-27 18:28:40,923 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-cdd088b6) (logid:81c638a6) End cleanup expired
> async-jobs
> 2019-08-27 18:28:46,252 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-df1af6ab) (logid:aa5aa04b) AutoScaling Monitor is
> running...
> 2019-08-27 18:28:46,491 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-c53610bd) (logid:e6acbb70) HostStatsCollector is
> running...
> 2019-08-27 18:28:46,731 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-956a1f24) (logid:10807b3f) StorageCollector is
> running...
> 2019-08-27 18:28:46,742 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-956a1f24) (logid:10807b3f) There is no secondary
> storage VM for secondary storage host Secondary
> 2019-08-27 18:28:50,906 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2536d816) (logid:166a57b0) Begin cleanup
> expired async-jobs
> 2019-08-27 18:28:50,907 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2536d816) (logid:166a57b0) End cleanup expired
> async-jobs
> [root@fr-kvm1 agent]#
>
>
> Nicolas.Vazquez@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message