cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Piotr Pisz" <pi...@piszki.pl>
Subject snapshot is not working
Date Mon, 09 Nov 2020 12:22:24 GMT
Hello,

 

I need help, snapshots are not working (see below), this is a new CS installation on Ceph
RBD. Besides, everything works fine, could you tell us what might be wrong?

 

Take snapshot: 544 failed

com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException:
org.libvirt.LibvirtException: XML error: missing storage pool source path

 

???

 

Best regards,

Piotr

 

 

2020-11-09 12:56:46,525 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-35:ctx-326b32af
job-1894) (logid:836f528d) Add job-1894 into job monitoring

2020-11-09 12:56:46,528 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (qtp1497845528-1209:ctx-a9fac85e
ctx-7d1c63e1) (logid:a5ddc58d) submit async job-1894, details: AsyncJobVO {id:1894, userId:
2, accountId: 2, instanceType: Snapshot, instanceId: 13, cmd: org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd,
cmdInfo: {"quiescevm":"false","httpmethod":"GET","ctxAccountId":"2","uuid":"82f3faf3-e09c-4eaf-886c-d81bad998b64","cmdEventType":"SNAPSHOT.CREATE","asyncBackup":"false","response":"json","ctxUserId":"2","volumeId":"9945a23b-a376-4f85-b378-9f7da8017dac","name":"test","ctxStartEventId":"4270","id":"13","ctxDetails":"{\"interface
com.cloud.storage.Volume\":\"9945a23b-a376-4f85-b378-9f7da8017dac\",\"interface com.cloud.storage.Snapshot\":\"82f3faf3-e09c-4eaf-886c-d81bad998b64\"}","_":"1604922798590"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
176206389407942, completeMsid: null, lastUpdated: null, lastPolled: null, created: null, removed:
null}

2020-11-09 12:56:46,528 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-35:ctx-326b32af
job-1894) (logid:db479fbc) Executing AsyncJobVO {id:1894, userId: 2, accountId: 2, instanceType:
Snapshot, instanceId: 13, cmd: org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd,
cmdInfo: {"quiescevm":"false","httpmethod":"GET","ctxAccountId":"2","uuid":"82f3faf3-e09c-4eaf-886c-d81bad998b64","cmdEventType":"SNAPSHOT.CREATE","asyncBackup":"false","response":"json","ctxUserId":"2","volumeId":"9945a23b-a376-4f85-b378-9f7da8017dac","name":"test","ctxStartEventId":"4270","id":"13","ctxDetails":"{\"interface
com.cloud.storage.Volume\":\"9945a23b-a376-4f85-b378-9f7da8017dac\",\"interface com.cloud.storage.Snapshot\":\"82f3faf3-e09c-4eaf-886c-d81bad998b64\"}","_":"1604922798590"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
176206389407942, completeMsid: null, lastUpdated: null, lastPolled: null, created: null, removed:
null}

2020-11-09 12:56:46,528 DEBUG [c.c.a.ApiServlet] (qtp1497845528-1209:ctx-a9fac85e ctx-7d1c63e1)
(logid:a5ddc58d) ===END===  10.89.11.31 -- GET  command=createSnapshot&response=json&volumeId=9945a23b-a376-4f85-b378-9f7da8017dac&quiescevm=false&asyncBackup=false&name=test&_=1604922798590

2020-11-09 12:56:46,549 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-35:ctx-326b32af
job-1894 ctx-f8b08221) (logid:db479fbc) Sync job-1897 execution on object VmWorkJobQueue.183

2020-11-09 12:56:46,850 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-13:null)
(logid:) Seq 1-8981303556883622697: Routing from 176206389408022

2020-11-09 12:56:46,851 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-6:null)
(logid:) Seq 1-8981303556883622697: MgmtId 176206389408022: Resp: Routing to peer

2020-11-09 12:56:47,086 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-3:null)
(logid:) Seq 43-2584503236157253953: Routing from 176206389408022

2020-11-09 12:56:47,126 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-14:null)
(logid:) Seq 43-2584503236157253953: MgmtId 176206389408022: Resp: Routing to peer

2020-11-09 12:56:47,131 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-15:null)
(logid:) Seq 46-3634967849241411607: Routing from 176206389408022

2020-11-09 12:56:47,133 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-10:null)
(logid:) Seq 46-3634967849241411607: MgmtId 176206389408022: Resp: Routing to peer

2020-11-09 12:56:47,181 DEBUG [c.c.s.StatsCollector] (StatsCollector-6:ctx-100fc604) (logid:fde56d09)
StorageCollector is running...

2020-11-09 12:56:47,184 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] (StatsCollector-6:ctx-100fc604)
(logid:fde56d09) getCommandHostDelegation: class com.cloud.agent.api.GetStorageStatsCommand

2020-11-09 12:56:47,184 DEBUG [c.c.h.XenServerGuru] (StatsCollector-6:ctx-100fc604) (logid:fde56d09)
We are returning the default host to execute commands because the command is not of Copy type.

2020-11-09 12:56:47,230 DEBUG [c.c.a.t.Request] (StatsCollector-6:ctx-100fc604) (logid:fde56d09)
Seq 58-4654188739910703033: Received:  { Ans: , MgmtId: 176206389407942, via: 58(s-10-VM),
Ver: v1, Flags: 10, { GetStorageStatsAnswer } }

2020-11-09 12:56:47,231 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] (StatsCollector-6:ctx-100fc604)
(logid:fde56d09) getCommandHostDelegation: class com.cloud.agent.api.GetStorageStatsCommand

2020-11-09 12:56:47,231 DEBUG [c.c.h.XenServerGuru] (StatsCollector-6:ctx-100fc604) (logid:fde56d09)
We are returning the default host to execute commands because the command is not of Copy type.

2020-11-09 12:56:47,232 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-6:ctx-100fc604)
(logid:fde56d09) Seq 37-1383449510532885592: Forwarding null to 176206389407982

2020-11-09 12:56:47,406 DEBUG [c.c.a.ApiServlet] (qtp1497845528-1280:ctx-b087af82) (logid:3a8983a6)
===START===  10.89.11.31 -- GET  command=listInfrastructure&response=json&_=1604922838197

2020-11-09 12:56:47,410 DEBUG [c.c.a.ApiServer] (qtp1497845528-1280:ctx-b087af82 ctx-052d26fb)
(logid:3a8983a6) CIDRs from which account 'Acct[c3dd8977-12ba-11eb-9854-a0423f44350c-admin]'
is allowed to perform API calls: 0.0.0.0/0,::/0

2020-11-09 12:56:47,416 DEBUG [c.c.a.ApiServlet] (qtp1497845528-1280:ctx-b087af82 ctx-052d26fb)
(logid:3a8983a6) ===END===  10.89.11.31 -- GET  command=listInfrastructure&response=json&_=1604922838197

2020-11-09 12:56:47,634 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-ccb57103)
(logid:56d3b696) Execute sync-queue item: SyncQueueItemVO {id:841, queueId: 601, contentType:
AsyncJob, contentId: 1897, lastProcessMsid: 176206389407942, lastprocessNumber: 5, lastProcessTime:
Mon Nov 09 12:56:47 CET 2020, created: Mon Nov 09 12:56:46 CET 2020}

2020-11-09 12:56:47,635 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-ccb57103)
(logid:56d3b696) Schedule queued job-1897

2020-11-09 12:56:47,637 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:1f589139) Add job-1897 into job monitoring

2020-11-09 12:56:47,640 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Executing AsyncJobVO {id:1897, userId: 2, accountId: 2,
instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkTakeVolumeSnapshot, cmdInfo:
rO0ABXNyACVjb20uY2xvdWQudm0uVm1Xb3JrVGFrZVZvbHVtZVNuYXBzaG90BL5gG4Li1c8CAAZaAAthc3luY0JhY2t1cFoACXF1aWVzY2VWbUwADGxvY2F0aW9uVHlwZXQAKUxjb20vY2xvdWQvc3RvcmFnZS9TbmFwc2hvdCRMb2NhdGlvblR5cGU7TAAIcG9saWN5SWR0ABBMamF2YS9sYW5nL0xvbmc7TAAKc25hcHNob3RJZHEAfgACTAAIdm9sdW1lSWRxAH4AAnhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAt3QAFFZvbHVtZUFwaVNlcnZpY2VJbXBsAABwc3IADmphdmEubGFuZy5Mb25nO4vkkMyPI98CAAFKAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAAAAAAAAAc3EAfgAHAAAAAAAAAA1zcQB-AAcAAAAAAAACIA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
176206389407942, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Nov
09 12:56:46 CET 2020, removed: null}

2020-11-09 12:56:47,640 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Run VM work job: com.cloud.vm.VmWorkTakeVolumeSnapshot
for VM 183, job origin: 1894

2020-11-09 12:56:47,642 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) Execute VM work job: com.cloud.vm.VmWorkTakeVolumeSnapshot{"volumeId":544,"policyId":0,"snapshotId":13,"quiesceVm":false,"asyncBackup":false,"userId":2,"accountId":2,"vmId":183,"handlerName":"VolumeApiServiceImpl"}

2020-11-09 12:56:47,675 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) getCommandHostDelegation: class org.apache.cloudstack.storage.command.CreateObjectCommand

2020-11-09 12:56:47,675 DEBUG [c.c.h.XenServerGuru] (Work-Job-Executor-26:ctx-15ef9a01 job-1894/job-1897
ctx-5fb6d0f8) (logid:db479fbc) We are returning the default host to execute commands because
the command is not of Copy type.

2020-11-09 12:56:47,676 DEBUG [c.c.a.m.ClusteredAgentAttache] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) Seq 40-1407656358530002013: Forwarding Seq
40-1407656358530002013:  { Cmd , MgmtId: 176206389407942, via: 40(srv-oz-r08-n08.cenagis.local),
Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"9945a23b-a376-4f85-b378-9f7da8017dac","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"22d1051a-86dc-3613-961d-f909245f5477","id":4,"poolType":"RBD","host":"rbd.cenagis.local","path":"rbd","port":6789,"url":"RBD://rbd.cenagis.local/rbd/?ROLE=Primary&STOREUUID=22d1051a-86dc-3613-961d-f909245f5477","isManaged":false}},"name":"ROOT-183","size":107374182400,"path":"9945a23b-a376-4f85-b378-9f7da8017dac","volumeId":544,"vmName":"i-2-183-VM","accountId":2,"format":"RAW","provisioningType":"THIN","id":544,"deviceId":0,"cacheMode":"WRITEBACK","hypervisorType":"KVM","directDownload":false},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"22d1051a-86dc-3613-961d-f909245f5477","id":4,"poolType":"RBD","host":"rbd.cenagis.local","path":"rbd","port":6789,"url":"RBD://rbd.cenagis.local/rbd/?ROLE=Primary&STOREUUID=22d1051a-86dc-3613-961d-f909245f5477","isManaged":false}},"vmName":"i-2-183-VM","name":"test","hypervisorType":"KVM","id":13,"quiescevm":false,"physicalSize":0}},"wait":0}}]
} to 176206389407982

2020-11-09 12:56:47,976 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-1:null)
(logid:) Seq 1-8603282663161211720: Routing from 176206389407982

2020-11-09 12:56:47,982 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-4:null)
(logid:) Seq 1-8981303556883622698: Routing from 176206389408022

2020-11-09 12:56:48,043 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-79a33e59)
(logid:4c5fdc64) Zone 1 is ready to launch console proxy

2020-11-09 12:56:48,150 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-c8a78ea1)
(logid:473dea1d) Zone 1 is ready to launch secondary storage VM

2020-11-09 12:56:48,190 DEBUG [c.c.a.t.Request] (StatsCollector-5:ctx-e6db6758) (logid:451965d6)
Seq 1-8552617167353293650: Received:  { Ans: , MgmtId: 176206389407942, via: 1(srv-oz-r01-n01.cenagis.local),
Ver: v1, Flags: 10, { GetVmStatsAnswer } }

2020-11-09 12:56:48,190 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-5:null)
(logid:) Seq 1-8603282663161211720: MgmtId 176206389407982: Resp: Routing to peer

2020-11-09 12:56:48,198 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-11:null)
(logid:) Seq 1-8981303556883622698: MgmtId 176206389408022: Resp: Routing to peer

2020-11-09 12:56:48,212 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq
40-1407656358530002013: Processing:  { Ans: , MgmtId: 176206389407942, via: 40, Ver: v1, Flags:
10, [{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"rbd/9945a23b-a376-4f85-b378-9f7da8017dac/f957d3a8-702a-45aa-9e42-5222c223ef19","id":0,"quiescevm":false,"physicalSize":0}},"result":true,"wait":0}}]
}

2020-11-09 12:56:48,212 DEBUG [c.c.a.t.Request] (Work-Job-Executor-26:ctx-15ef9a01 job-1894/job-1897
ctx-5fb6d0f8) (logid:db479fbc) Seq 40-1407656358530002013: Received:  { Ans: , MgmtId: 176206389407942,
via: 40(srv-oz-r08-n08.cenagis.local), Ver: v1, Flags: 10, { CreateObjectAnswer } }

2020-11-09 12:56:48,226 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-5:ctx-e6db6758)
(logid:451965d6) Seq 22-6063533948300963066: Forwarding null to 176206389407982

2020-11-09 12:56:48,248 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) copyAsync inspecting src type SNAPSHOT copyAsync
inspecting dest type SNAPSHOT

2020-11-09 12:56:48,249 DEBUG [c.c.a.ApiServlet] (qtp1497845528-1209:ctx-4e81cf12) (logid:2f3c90fe)
===START===  10.89.11.31 -- GET  command=listHosts&response=json&type=routing&listAll=true&page=1&pagesize=20&_=1604922838198

2020-11-09 12:56:48,253 DEBUG [c.c.a.ApiServer] (qtp1497845528-1209:ctx-4e81cf12 ctx-0b68411a)
(logid:2f3c90fe) CIDRs from which account 'Acct[c3dd8977-12ba-11eb-9854-a0423f44350c-admin]'
is allowed to perform API calls: 0.0.0.0/0,::/0

2020-11-09 12:56:48,254 WARN  [c.c.a.d.ParamGenericValidationWorker] (qtp1497845528-1209:ctx-4e81cf12
ctx-0b68411a) (logid:2f3c90fe) Received unknown parameters for command listHosts. Unknown
parameters : listall

2020-11-09 12:56:48,254 DEBUG [c.c.a.q.QueryManagerImpl] (qtp1497845528-1209:ctx-4e81cf12
ctx-0b68411a) (logid:2f3c90fe) >>>Searching for hosts>>>

2020-11-09 12:56:48,259 DEBUG [c.c.a.q.QueryManagerImpl] (qtp1497845528-1209:ctx-4e81cf12
ctx-0b68411a) (logid:2f3c90fe) >>>Generating Response>>>

2020-11-09 12:56:48,293 DEBUG [c.c.a.ApiServlet] (qtp1497845528-1209:ctx-4e81cf12 ctx-0b68411a)
(logid:2f3c90fe) ===END===  10.89.11.31 -- GET  command=listHosts&response=json&type=routing&listAll=true&page=1&pagesize=20&_=1604922838198

2020-11-09 12:56:48,298 ERROR [o.a.c.s.e.DefaultEndPointSelector] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) IR24 select BACKUPSNAPSHOT from primary to
secondary 13 dest=13

2020-11-09 12:56:48,302 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) getCommandHostDelegation: class org.apache.cloudstack.storage.command.CopyCommand

2020-11-09 12:56:48,303 DEBUG [c.c.a.m.ClusteredAgentAttache] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) Seq 49-6537256334104996965: Forwarding Seq
49-6537256334104996965:  { Cmd , MgmtId: 176206389407942, via: 49(srv-oz-r11-n11.cenagis.local),
Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"rbd/9945a23b-a376-4f85-b378-9f7da8017dac/f957d3a8-702a-45aa-9e42-5222c223ef19","volume":{"uuid":"9945a23b-a376-4f85-b378-9f7da8017dac","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"22d1051a-86dc-3613-961d-f909245f5477","id":4,"poolType":"RBD","host":"rbd.cenagis.local","path":"rbd","port":6789,"url":"RBD://rbd.cenagis.local/rbd/?ROLE=Primary&STOREUUID=22d1051a-86dc-3613-961d-f909245f5477","isManaged":false}},"name":"ROOT-183","size":107374182400,"path":"9945a23b-a376-4f85-b378-9f7da8017dac","volumeId":544,"vmName":"i-2-183-VM","accountId":2,"format":"RAW","provisioningType":"THIN","id":544,"deviceId":0,"cacheMode":"WRITEBACK","hypervisorType":"KVM","directDownload":false},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"22d1051a-86dc-3613-961d-f909245f5477","id":4,"poolType":"RBD","host":"rbd.cenagis.local","path":"rbd","port":6789,"url":"RBD://rbd.cenagis.local/rbd/?ROLE=Primary&STOREUUID=22d1051a-86dc-3613-961d-f909245f5477","isManaged":false}},"vmName":"i-2-183-VM","name":"test","hypervisorType":"KVM","id":13,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/544","volume":{"uuid":"9945a23b-a376-4f85-b378-9f7da8017dac","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"22d1051a-86dc-3613-961d-f909245f5477","id":4,"poolType":"RBD","host":"rbd.cenagis.local","path":"rbd","port":6789,"url":"RBD://rbd.cenagis.local/rbd/?ROLE=Primary&STOREUUID=22d1051a-86dc-3613-961d-f909245f5477","isManaged":false}},"name":"ROOT-183","size":107374182400,"path":"9945a23b-a376-4f85-b378-9f7da8017dac","volumeId":544,"vmName":"i-2-183-VM","accountId":2,"format":"RAW","provisioningType":"THIN","id":544,"deviceId":0,"cacheMode":"WRITEBACK","hypervisorType":"KVM","directDownload":false},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://sys2.cenagis.local/","_role":"Image"}},"vmName":"i-2-183-VM","name":"test","hypervisorType":"KVM","id":13,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"fullSnapshot":"true"},"options2":{},"wait":216000}}]
} to 176206389407982

2020-11-09 12:56:48,364 DEBUG [c.c.a.t.Request] (AgentManager-Handler-7:null) (logid:) Seq
49-6537256334104996965: Processing:  { Ans: , MgmtId: 176206389407942, via: 49, Ver: v1, Flags:
10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException:
org.libvirt.LibvirtException: XML error: missing storage pool source path","wait":0}}] }

2020-11-09 12:56:48,364 DEBUG [c.c.a.t.Request] (Work-Job-Executor-26:ctx-15ef9a01 job-1894/job-1897
ctx-5fb6d0f8) (logid:db479fbc) Seq 49-6537256334104996965: Received:  { Ans: , MgmtId: 176206389407942,
via: 49(srv-oz-r11-n11.cenagis.local), Ver: v1, Flags: 10, { CopyCmdAnswer } }

2020-11-09 12:56:48,373 DEBUG [c.c.s.s.SnapshotManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) Failed to create snapshotcom.cloud.utils.exception.CloudRuntimeException:
org.libvirt.LibvirtException: XML error: missing storage pool source path

2020-11-09 12:56:48,373 DEBUG [c.c.r.ResourceLimitManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) Updating resource Type = snapshot count for
Account = 2 Operation = decreasing Amount = 1

2020-11-09 12:56:48,375 DEBUG [c.c.r.ResourceLimitManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) Updating resource Type = secondary_storage
count for Account = 2 Operation = decreasing Amount = 107374182400

2020-11-09 12:56:48,377 ERROR [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) Take snapshot: 544 failed

com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException:
org.libvirt.LibvirtException: XML error: missing storage pool source path

        at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:301)

        at org.apache.cloudstack.storage.snapshot.DefaultSnapshotStrategy.backupSnapshot(DefaultSnapshotStrategy.java:171)

        at com.cloud.storage.snapshot.SnapshotManagerImpl.backupSnapshotToSecondary(SnapshotManagerImpl.java:1213)

        at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1164)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.base/java.lang.reflect.Method.invoke(Method.java:566)

        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)

        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)

        at com.sun.proxy.$Proxy205.takeSnapshot(Unknown Source)

        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:2073)

        at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:2541)

        at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:3465)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.base/java.lang.reflect.Method.invoke(Method.java:566)

        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)

        at com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:3471)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.base/java.lang.reflect.Method.invoke(Method.java:566)

        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)

        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)

        at com.sun.proxy.$Proxy212.handleVmWorkJob(Unknown Source)

        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)

        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:603)

        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)

        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)

        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)

        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)

        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)

        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:551)

        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

        at java.base/java.lang.Thread.run(Thread.java:834)

2020-11-09 12:56:48,378 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) Invocation exception, caused by: com.cloud.utils.exception.CloudRuntimeException:
com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: XML error:
missing storage pool source path

2020-11-09 12:56:48,378 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897 ctx-5fb6d0f8) (logid:db479fbc) Rethrow exception com.cloud.utils.exception.CloudRuntimeException:
com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: XML error:
missing storage pool source path

2020-11-09 12:56:48,378 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Done with run of VM work job: com.cloud.vm.VmWorkTakeVolumeSnapshot
for VM 183, job origin: 1894

2020-11-09 12:56:48,378 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Unable to complete AsyncJobVO {id:1897, userId: 2, accountId:
2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkTakeVolumeSnapshot, cmdInfo:
rO0ABXNyACVjb20uY2xvdWQudm0uVm1Xb3JrVGFrZVZvbHVtZVNuYXBzaG90BL5gG4Li1c8CAAZaAAthc3luY0JhY2t1cFoACXF1aWVzY2VWbUwADGxvY2F0aW9uVHlwZXQAKUxjb20vY2xvdWQvc3RvcmFnZS9TbmFwc2hvdCRMb2NhdGlvblR5cGU7TAAIcG9saWN5SWR0ABBMamF2YS9sYW5nL0xvbmc7TAAKc25hcHNob3RJZHEAfgACTAAIdm9sdW1lSWRxAH4AAnhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAt3QAFFZvbHVtZUFwaVNlcnZpY2VJbXBsAABwc3IADmphdmEubGFuZy5Mb25nO4vkkMyPI98CAAFKAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAAAAAAAAAc3EAfgAHAAAAAAAAAA1zcQB-AAcAAAAAAAACIA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
176206389407942, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Nov
09 12:56:46 CET 2020, removed: null}, job origin:1894

com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException:
org.libvirt.LibvirtException: XML error: missing storage pool source path

        at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:301)

        at org.apache.cloudstack.storage.snapshot.DefaultSnapshotStrategy.backupSnapshot(DefaultSnapshotStrategy.java:171)

        at com.cloud.storage.snapshot.SnapshotManagerImpl.backupSnapshotToSecondary(SnapshotManagerImpl.java:1213)

        at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1164)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.base/java.lang.reflect.Method.invoke(Method.java:566)

        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)

        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)

        at com.sun.proxy.$Proxy205.takeSnapshot(Unknown Source)

        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:2073)

        at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:2541)

        at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:3465)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.base/java.lang.reflect.Method.invoke(Method.java:566)

        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)

        at com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:3471)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.base/java.lang.reflect.Method.invoke(Method.java:566)

        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)

        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)

        at com.sun.proxy.$Proxy212.handleVmWorkJob(Unknown Source)

        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)

        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:603)

        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)

        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)

        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)

        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)

        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)

        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:551)

        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

        at java.base/java.lang.Thread.run(Thread.java:834)

2020-11-09 12:56:48,380 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Complete async job-1897, jobStatus: FAILED, resultCode:
0, result: rO0ABXNyAC9jb20uY2xvdWQudXRpbHMuZXhjZXB0aW9uLkNsb3VkUnVudGltZUV4Y2VwdGlvbgAAAABWTU9yAwABSQALY3NFcnJvckNvZGV4cgAaamF2YS5sYW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0fPho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAKc3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRFeGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AHpjb20uY2xvdWQudXRpbHMuZXhjZXB0aW9uLkNsb3VkUnVudGltZUV4Y2VwdGlvbjogb3JnLmxpYnZpcnQuTGlidmlydEV4Y2VwdGlvbjogWE1MIGVycm9yOiBtaXNzaW5nIHN0b3JhZ2UgcG9vbCBzb3VyY2UgcGF0aHVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAMHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH4ABUwADmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfgAFTAAKbW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAABLXQAA2FwcHQAOm9yZy5hcGFjaGUuY2xvdWRzdGFjay5zdG9yYWdlLnNuYXBzaG90LlNuYXBzaG90U2VydmljZUltcGx0ABhTbmFwc2hvdFNlcnZpY2VJbXBsLmphdmF0AA5iYWNrdXBTbmFwc2hvdHBwc3EAfgAMAQAAAKtxAH4ADnQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5zdG9yYWdlLnNuYXBzaG90LkRlZmF1bHRTbmFwc2hvdFN0cmF0ZWd5dAAcRGVmYXVsdFNuYXBzaG90U3RyYXRlZ3kuamF2YXEAfgARcHBzcQB-AAwBAAAEvXEAfgAOdAAuY29tLmNsb3VkLnN0b3JhZ2Uuc25hcHNob3QuU25hcHNob3RNYW5hZ2VySW1wbHQAGFNuYXBzaG90TWFuYWdlckltcGwuamF2YXQAGWJhY2t1cFNuYXBzaG90VG9TZWNvbmRhcnlwcHNxAH4ADAEAAASMcQB-AA5xAH4AFnEAfgAXdAAMdGFrZVNuYXBzaG90cHBzcQB-AAwC_____nB0AC1qZGsuaW50ZXJuYWwucmVmbGVjdC5OYXRpdmVNZXRob2RBY2Nlc3NvckltcGx0AB1OYXRpdmVNZXRob2RBY2Nlc3NvckltcGwuamF2YXQAB2ludm9rZTB0AAlqYXZhLmJhc2V0AAYxMS4wLjhzcQB-AAwCAAAAPnBxAH4AHHEAfgAddAAGaW52b2tlcQB-AB9xAH4AIHNxAH4ADAIAAAArcHQAMWpkay5pbnRlcm5hbC5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AInEAfgAfcQB-ACBzcQB-AAwCAAACNnB0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAicQB-AB9xAH4AIHNxAH4ADAEAAAFYcQB-AA50AChvcmcuc3ByaW5nZnJhbWV3b3JrLmFvcC5zdXBwb3J0LkFvcFV0aWxzdAANQW9wVXRpbHMuamF2YXQAHmludm9rZUpvaW5wb2ludFVzaW5nUmVmbGVjdGlvbnBwc3EAfgAMAQAAAMZxAH4ADnQAPG9yZy5zcHJpbmdmcmFtZXdvcmsuYW9wLmZyYW1ld29yay5SZWZsZWN0aXZlTWV0aG9kSW52b2NhdGlvbnQAH1JlZmxlY3RpdmVNZXRob2RJbnZvY2F0aW9uLmphdmF0AA9pbnZva2VKb2lucG9pbnRwcHNxAH4ADAEAAACjcQB-AA5xAH4ALnEAfgAvdAAHcHJvY2VlZHBwc3EAfgAMAQAAAF9xAH4ADnQAP29yZy5zcHJpbmdmcmFtZXdvcmsuYW9wLmludGVyY2VwdG9yLkV4cG9zZUludm9jYXRpb25JbnRlcmNlcHRvcnQAIEV4cG9zZUludm9jYXRpb25JbnRlcmNlcHRvci5qYXZhcQB-ACJwcHNxAH4ADAEAAAC6cQB-AA5xAH4ALnEAfgAvcQB-ADJwcHNxAH4ADAEAAADUcQB-AA50ADRvcmcuc3ByaW5nZnJhbWV3b3JrLmFvcC5mcmFtZXdvcmsuSmRrRHluYW1pY0FvcFByb3h5dAAXSmRrRHluYW1pY0FvcFByb3h5LmphdmFxAH4AInBwc3EAfgAMAP____9wdAAXY29tLnN1bi5wcm94eS4kUHJveHkyMDVwcQB-ABpwcHNxAH4ADAEAAAgZcQB-AA50ADZvcmcuYXBhY2hlLmNsb3Vkc3RhY2suc3RvcmFnZS52b2x1bWUuVm9sdW1lU2VydmljZUltcGx0ABZWb2x1bWVTZXJ2aWNlSW1wbC5qYXZhcQB-ABpwcHNxAH4ADAEAAAntcQB-AA50ACZjb20uY2xvdWQuc3RvcmFnZS5Wb2x1bWVBcGlTZXJ2aWNlSW1wbHQAGVZvbHVtZUFwaVNlcnZpY2VJbXBsLmphdmF0AB1vcmNoZXN0cmF0ZVRha2VWb2x1bWVTbmFwc2hvdHBwc3EAfgAMAQAADYlxAH4ADnEAfgBAcQB-AEFxAH4AQnBwc3EAfgAMAv____5wcQB-ABxxAH4AHXEAfgAecQB-AB9xAH4AIHNxAH4ADAIAAAA-cHEAfgAccQB-AB1xAH4AInEAfgAfcQB-ACBzcQB-AAwCAAAAK3BxAH4AJHEAfgAlcQB-ACJxAH4AH3EAfgAgc3EAfgAMAgAAAjZwcQB-ACdxAH4AKHEAfgAicQB-AB9xAH4AIHNxAH4ADAEAAABrcQB-AA50ACJjb20uY2xvdWQudm0uVm1Xb3JrSm9iSGFuZGxlclByb3h5dAAaVm1Xb3JrSm9iSGFuZGxlclByb3h5LmphdmF0AA9oYW5kbGVWbVdvcmtKb2JwcHNxAH4ADAEAAA2PcQB-AA5xAH4AQHEAfgBBcQB-AEtwcHNxAH4ADAL____-cHEAfgAccQB-AB1xAH4AHnEAfgAfcQB-ACBzcQB-AAwCAAAAPnBxAH4AHHEAfgAdcQB-ACJxAH4AH3EAfgAgc3EAfgAMAgAAACtwcQB-ACRxAH4AJXEAfgAicQB-AB9xAH4AIHNxAH4ADAIAAAI2cHEAfgAncQB-AChxAH4AInEAfgAfcQB-ACBzcQB-AAwBAAABWHEAfgAOcQB-ACpxAH4AK3EAfgAscHBzcQB-AAwBAAAAxnEAfgAOcQB-AC5xAH4AL3EAfgAwcHBzcQB-AAwBAAAAo3EAfgAOcQB-AC5xAH4AL3EAfgAycHBzcQB-AAwBAAAAX3EAfgAOcQB-ADRxAH4ANXEAfgAicHBzcQB-AAwBAAAAunEAfgAOcQB-AC5xAH4AL3EAfgAycHBzcQB-AAwBAAAA1HEAfgAOcQB-ADhxAH4AOXEAfgAicHBzcQB-AAwA_____3B0ABdjb20uc3VuLnByb3h5LiRQcm94eTIxMnBxAH4AS3Bwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS5WbVdvcmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icHBzcQB-AAwBAAACW3EAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0cHBzcQB-AAwBAAAAMXEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxlLmphdmF0AANydW5wcHNxAH4ADAEAAAA4cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TWFuYWdlZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABncQB-AA50AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AGd0AA9jYWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA1cQB-AA5xAH4AanEAfgBndAAOcnVuV2l0aENvbnRleHRwcHNxAH4ADAEAAAAucQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4AY3EAfgBkcHBzcQB-AAwBAAACJ3EAfgAOcQB-AF5xAH4AX3EAfgBkcHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeGVjdXRvcnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AaHEAfgAfcQB-ACBzcQB-AAwCAAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay5qYXZhcQB-AGRxAH4AH3EAfgAgc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH4AH3EAfgAgc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9yJFdvcmtlcnEAfgB5cQB-AGRxAH4AH3EAfgAgc3EAfgAMAgAAA0JwdAAQamF2YS5sYW5nLlRocmVhZHQAC1RocmVhZC5qYXZhcQB-AGRxAH4AH3EAfgAgc3IAH2phdmEudXRpbC5Db2xsZWN0aW9ucyRFbXB0eUxpc3R6uBe0PKee3gIAAHhweAAAEJp3CAAAAAAAAAAAeA

2020-11-09 12:56:48,380 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Publish async job-1897 complete on message bus

2020-11-09 12:56:48,380 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Wake up jobs related to job-1897

2020-11-09 12:56:48,380 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Update db status for job-1897

2020-11-09 12:56:48,381 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Wake up jobs joined with job-1897 and disjoin all subjobs
created from job- 1897

2020-11-09 12:56:48,385 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Done executing com.cloud.vm.VmWorkTakeVolumeSnapshot for
job-1897

2020-11-09 12:56:48,385 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-26:ctx-15ef9a01
job-1894/job-1897) (logid:db479fbc) Remove job-1897 from job monitoring

2020-11-09 12:56:48,392 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-35:ctx-326b32af
job-1894) (logid:db479fbc) Complete async job-1894, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
to create snapshot due to an internal error creating snapshot for volume 9945a23b-a376-4f85-b378-9f7da8017dac"}

2020-11-09 12:56:48,392 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-35:ctx-326b32af
job-1894) (logid:db479fbc) Publish async job-1894 complete on message bus


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message