From users-return-34536-apmail-cloudstack-users-archive=cloudstack.apache.org@cloudstack.apache.org Sun Jun 14 11:44:10 2020 Return-Path: X-Original-To: apmail-cloudstack-users-archive@www.apache.org Delivered-To: apmail-cloudstack-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by minotaur.apache.org (Postfix) with SMTP id BD00119295 for ; Sun, 14 Jun 2020 11:44:09 +0000 (UTC) Received: (qmail 4978 invoked by uid 500); 14 Jun 2020 11:44:06 -0000 Delivered-To: apmail-cloudstack-users-archive@cloudstack.apache.org Received: (qmail 4895 invoked by uid 500); 14 Jun 2020 11:44:06 -0000 Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@cloudstack.apache.org Delivered-To: mailing list users@cloudstack.apache.org Received: (qmail 4873 invoked by uid 99); 14 Jun 2020 11:44:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 14 Jun 2020 11:44:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 6223CC137F for ; Sun, 14 Jun 2020 11:44:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id P2yRootayhTe for ; Sun, 14 Jun 2020 11:43:59 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.166.48; helo=mail-io1-f48.google.com; envelope-from=mdvlinquest@gmail.com; receiver= Received: from mail-io1-f48.google.com (mail-io1-f48.google.com [209.85.166.48]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 1146EBB802 for ; Sun, 14 Jun 2020 11:43:59 +0000 (UTC) Received: by mail-io1-f48.google.com with SMTP id r77so14846271ior.3 for ; Sun, 14 Jun 2020 04:43:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=sj9+1XsKUMkDccjbdJtSGRYmPWt5utuSwQC4p2/0jBM=; b=ckulxne4xhMGv19oEQcEJ+UhUQ4FIxwz7cxzwi7E1VrSzv8v5vzj+pWW5joTQ4YEVF I9mei6NP2ea80qjAPUErdBJvBKdsLbZloFP403MF8OpRzCalzt/S3ce251nTFRCbAtP4 PvnlQWyOL+YJEEnpvjnC4Uk7TLJNmpfOhllLCxiS0RaGz7PMUQqUnHa5B1MpgiJi3vT/ a2AATAjalfnvZtP9HjuqBzhn1t0p42XTIofGVZXpcXBsyYVl7PaUR/ELcocAriUbui6G +UacX/X43M07HpGvygH6aC9HChO69UYSRiOHbWvwEa41ycl/U6R9N6MJN/3GexfW8P9J D7iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=sj9+1XsKUMkDccjbdJtSGRYmPWt5utuSwQC4p2/0jBM=; b=IZZzkJtVNjVzIG0cLpmg+F7yvwfumP0lwyEzqu4V7SRsrvbxtRZKPPMPsoRlbYh+4h Kp9mhVi2QhuoJc0TJ9jgmADBoHqc4wUDoG++X7w9BznhiNaLcrOWJ9ykvlMhJuo0ZyAs kAfpHCIPUdSlpZSTErK5+NUl2MiKnkxScBF465AYFdQfmwrZvzFze9/LTwu5q8ArGRJW MLjBCwTwscocXR5hpvzRet12ziQwbsAbTlnyHN/tNkZijLSAYKJVz3taq2knFzmIdSS9 4IkxtHC3/yj87x0xkZS2iYcXhUmFtzqxqq/rxzNIG6YR0JdFPkayPla6fvue+fJAKWsB 0D8w== X-Gm-Message-State: AOAM532tlgTAlVdSxqDzei+uLhZkbBg9125EgUd0p46gs3s75l5cl/Rq WM3QAQNSVdlaxgdzk8NICyTLQ3OfX6uor0hBNbL07TYS X-Google-Smtp-Source: ABdhPJz6kA0W4uLGntnMmHGr6jScr9mPDLdJMkC0PpXHj15tWR6e+tPMM2zbZho8rZ88p+aJmlyaAriGa4Tiirzdnqc= X-Received: by 2002:a02:cccd:: with SMTP id k13mr16247988jaq.135.1592135038111; Sun, 14 Jun 2020 04:43:58 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Fariborz Navidan Date: Sun, 14 Jun 2020 16:13:47 +0430 Message-ID: Subject: Re: Agent is unable to apply SG rules on new VMs. To: users@cloudstack.apache.org Content-Type: multipart/alternative; boundary="0000000000006194ca05a809d36d" --0000000000006194ca05a809d36d Content-Type: text/plain; charset="UTF-8" I'm now sure how it was happening. However. rebooting the KVM host had put it to work but I think it can happen again in future. On Sun, Jun 14, 2020 at 1:51 AM Andrija Panic wrote: > https://github.com/apache/cloudstack/pull/4124 > > On Sat, 13 Jun 2020, 18:25 Fariborz Navidan, > wrote: > > > Any idea how to fix? > > > > On Sat, Jun 13, 2020 at 7:07 PM Fariborz Navidan > > wrote: > > > > > Hello, > > > > > > I'm running CS 4.14 with KVM hypervisor. I have noticed that default SG > > > rules are not applied on the new VMs. I found errors in agent log which > > > state that it cannot apply default network rules on the for the VM > > > instance. Bellow what I have in agent log while creating new VM. Sorry > > for > > > posting the log here. > > > > > > [root@fr-kvm1 ~]# iptables --list | grep '433' > > > [root@fr-kvm1 ~]# cat /var/log/cloudstack/agent/security_groups.log > > > > > > [root@fr-kvm1 ~]# cat /var/log/cloudstack/agent/security_group.log > > > > > > [root@fr-kvm1 ~]# cat /var/log/cloudstack/agent/agent.log > > > > > > 2020-06-13 14:25:19,832 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:065cc7d6) Trying to fetch storage > > pool > > > 546baecf-c3cf-4180-bb1f-2d2faad501cf from libvirt > > > 2020-06-13 14:25:19,858 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:065cc7d6) Asking libvirt to > refresh > > > storage pool 546baecf-c3cf-4180-bb1f-2d2faad501cf > > > 2020-06-13 14:25:19,909 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-5:null) (logid:065cc7d6) Trying to fetch storage > > pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed from libvirt > > > 2020-06-13 14:25:19,912 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-5:null) (logid:065cc7d6) Asking libvirt to > refresh > > > storage pool adbffc6e-55e4-385e-987a-6aca3b0880ed > > > 2020-06-13 14:25:58,228 WARN > > > [resource.virtualnetwork.VirtualRoutingResource] > > > (agentRequest-Handler-1:null) (logid:3659aedc) Expected 1 answers while > > > executing DhcpEntryCommand but received 2 > > > 2020-06-13 14:25:58,820 WARN > > > [resource.virtualnetwork.VirtualRoutingResource] > > > (agentRequest-Handler-2:null) (logid:3659aedc) Expected 1 answers while > > > executing SavePasswordCommand but received 2 > > > 2020-06-13 14:25:59,718 WARN > > > [resource.virtualnetwork.VirtualRoutingResource] > > > (agentRequest-Handler-2:null) (logid:3659aedc) Expected 1 answers while > > > executing VmDataCommand but received 2 > > > 2020-06-13 14:25:59,788 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:3659aedc) Trying to fetch storage > > pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed from libvirt > > > 2020-06-13 14:25:59,809 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:3659aedc) Trying to fetch storage > > pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed from libvirt > > > 2020-06-13 14:25:59,822 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:3659aedc) Creating volume > > > 57d91afa-9a56-4a0c-8936-9335679df804 from template > > > 8ef6a817-c50b-4ac5-9589-a88a99275a5f in pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed (NetworkFilesystem) with size > > > 16106127360 > > > 2020-06-13 14:25:59,825 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:3659aedc) Attempting to create > > volume > > > 57d91afa-9a56-4a0c-8936-9335679df804 (NetworkFilesystem) in pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed with size 7516192768 > > > 2020-06-13 14:26:02,607 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-5:null) (logid:3659aedc) Trying to fetch storage > > pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed from libvirt > > > 2020-06-13 14:26:02,643 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-5:null) (logid:3659aedc) Trying to fetch storage > > pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed from libvirt > > > 2020-06-13 14:26:02,766 WARN [kvm.resource.LibvirtKvmAgentHook] > > > (agentRequest-Handler-5:null) (logid:3659aedc) Groovy script > > > '/etc/cloudstack/agent/hooks/libvirt-vm-xml-transformer.groovy' is not > > > available. Transformations will not be applied. > > > 2020-06-13 14:26:02,766 WARN [kvm.resource.LibvirtKvmAgentHook] > > > (agentRequest-Handler-5:null) (logid:3659aedc) Groovy scripting engine > is > > > not initialized. Data transformation skipped. > > > 2020-06-13 14:26:02,928 WARN [kvm.resource.LibvirtKvmAgentHook] > > > (agentRequest-Handler-5:null) (logid:3659aedc) Groovy script > > > '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not > > > available. Transformations will not be applied. > > > 2020-06-13 14:26:02,929 WARN [kvm.resource.LibvirtKvmAgentHook] > > > (agentRequest-Handler-5:null) (logid:3659aedc) Groovy scripting engine > is > > > not initialized. Data transformation skipped. > > > 2020-06-13 14:26:02,929 ERROR [kvm.resource.LibvirtComputingResource] > > > (agentRequest-Handler-5:null) (logid:3659aedc) Unable to apply default > > > network rule for nic cloudbr0 for VM i-23-433-VM > > > 2020-06-13 14:26:05,771 ERROR [kvm.resource.LibvirtComputingResource] > > > (agentRequest-Handler-1:null) (logid:2c66b1bd) Unable to apply default > > > network rule for nic cloudbr0 for VM i-23-433-VM > > > 2020-06-13 14:26:05,772 WARN > > > [resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper] > > > (agentRequest-Handler-1:null) (logid:2c66b1bd) Failed to program > default > > > network rules for vm i-23-433-VM > > > 2020-06-13 14:26:09,241 ERROR [kvm.resource.LibvirtComputingResource] > > > (agentRequest-Handler-2:null) (logid:e7718d11) Unable to apply default > > > network rule for nic cloudbr0 for VM i-23-433-VM > > > 2020-06-13 14:26:09,241 WARN > > > [resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper] > > > (agentRequest-Handler-2:null) (logid:e7718d11) Failed to program > default > > > network rules for vm i-23-433-VM > > > 2020-06-13 14:26:09,291 ERROR [kvm.resource.LibvirtComputingResource] > > > (agentRequest-Handler-4:null) (logid:d8b7f1cc) Unable to apply default > > > network rule for nic cloudbr0 for VM i-23-433-VM > > > 2020-06-13 14:26:09,291 WARN > > > [resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper] > > > (agentRequest-Handler-4:null) (logid:d8b7f1cc) Failed to program > default > > > network rules for vm i-23-433-VM > > > 2020-06-13 14:26:12,069 ERROR [kvm.resource.LibvirtComputingResource] > > > (agentRequest-Handler-3:null) (logid:94699055) Unable to apply default > > > network rule for nic cloudbr0 for VM i-23-433-VM > > > 2020-06-13 14:26:12,069 WARN > > > [resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper] > > > (agentRequest-Handler-3:null) (logid:94699055) Failed to program > default > > > network rules for vm i-23-433-VM > > > 2020-06-13 14:26:20,095 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-1:null) (logid:e4f546a3) Trying to fetch storage > > pool > > > 546baecf-c3cf-4180-bb1f-2d2faad501cf from libvirt > > > 2020-06-13 14:26:20,117 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-1:null) (logid:e4f546a3) Asking libvirt to > refresh > > > storage pool 546baecf-c3cf-4180-bb1f-2d2faad501cf > > > 2020-06-13 14:26:20,167 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-2:null) (logid:e4f546a3) Trying to fetch storage > > pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed from libvirt > > > 2020-06-13 14:26:20,169 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-2:null) (logid:e4f546a3) Asking libvirt to > refresh > > > storage pool adbffc6e-55e4-385e-987a-6aca3b0880ed > > > 2020-06-13 14:27:20,335 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-3:null) (logid:ca0db075) Trying to fetch storage > > pool > > > 546baecf-c3cf-4180-bb1f-2d2faad501cf from libvirt > > > 2020-06-13 14:27:20,357 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-3:null) (logid:ca0db075) Asking libvirt to > refresh > > > storage pool 546baecf-c3cf-4180-bb1f-2d2faad501cf > > > 2020-06-13 14:27:20,403 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-1:null) (logid:ca0db075) Trying to fetch storage > > pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed from libvirt > > > 2020-06-13 14:27:20,405 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-1:null) (logid:ca0db075) Asking libvirt to > refresh > > > storage pool adbffc6e-55e4-385e-987a-6aca3b0880ed > > > 2020-06-13 14:28:20,575 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-5:null) (logid:c6b61240) Trying to fetch storage > > pool > > > 546baecf-c3cf-4180-bb1f-2d2faad501cf from libvirt > > > 2020-06-13 14:28:20,600 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-5:null) (logid:c6b61240) Asking libvirt to > refresh > > > storage pool 546baecf-c3cf-4180-bb1f-2d2faad501cf > > > 2020-06-13 14:28:20,651 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-3:null) (logid:c6b61240) Trying to fetch storage > > pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed from libvirt > > > 2020-06-13 14:28:20,653 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-3:null) (logid:c6b61240) Asking libvirt to > refresh > > > storage pool adbffc6e-55e4-385e-987a-6aca3b0880ed > > > 2020-06-13 14:29:20,819 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:aba4ed7e) Trying to fetch storage > > pool > > > 546baecf-c3cf-4180-bb1f-2d2faad501cf from libvirt > > > 2020-06-13 14:29:20,840 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:aba4ed7e) Asking libvirt to > refresh > > > storage pool 546baecf-c3cf-4180-bb1f-2d2faad501cf > > > 2020-06-13 14:29:20,892 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-5:null) (logid:aba4ed7e) Trying to fetch storage > > pool > > > adbffc6e-55e4-385e-987a-6aca3b0880ed from libvirt > > > 2020-06-13 14:29:20,893 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-5:null) (logid:aba4ed7e) Asking libvirt to > refresh > > > storage pool adbffc6e-55e4-385e-987a-6aca3b0880ed > > > 2020-06-13 14:29:32,713 WARN > > > [resource.virtualnetwork.VirtualRoutingResource] > > > (agentRequest-Handler-2:null) (logid:03322b65) Expected 1 answers while > > > executing SetMonitorServiceCommand but received 3 > > > 2020-06-13 14:29:32,761 INFO > > > [resource.virtualnetwork.VirtualRoutingResource] > > > (agentRequest-Handler-4:null) (logid:a12b7894) Fetching health check > > result > > > for 169.254.1.154 and executing fresh checks: false > > > > > > --0000000000006194ca05a809d36d--