cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nux! <...@li.nux.ro>
Subject Re: Upgrade to Cloudstack 4.11.2 fails *AGAIN*
Date Wed, 22 May 2019 15:34:04 GMT
Eric,

I think you can get away with just downgrading to stock qemu-kvm packages (or patch your cloudstack
installation).
BTW qemu-kvm-ev is not part of stock CentOS for a reason, SIG packages don't get the same
level of testing and QA.

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

----- Original Message -----
> From: "Eric Lee Green" <eric.lee.green@gmail.com>
> To: "users" <users@cloudstack.apache.org>
> Sent: Wednesday, 22 May, 2019 15:33:15
> Subject: Re: Upgrade to Cloudstack 4.11.2 fails *AGAIN*

> Okay. This makes sense.
> 
> And people wonder why Amazon decided to make their own Linux rather than
> use Centos and why Ubuntu has seized huge market share from Red Hat in
> the past few years. SIGH.
> 
> Downgrading my CentOS is not going to be easy. There were security
> patches in the latest CentOS that I am required to have. It would
> actually be easier to just switch to Ubuntu at this point since we're
> talking a complete re-install anyhow. SIGH. Thank you, Red Hat Software.
> I hope IBM fixes them, but I have my doubts.
> 
> On 5/22/2019 1:05 AM, Andrija Panic wrote:
>> Hi Eric, all,
>>
>> I believe you might be hitting this one - issues on latest CentOS 7.6:
>> https://github.com/apache/cloudstack/pull/3333 due to changes in the OS
>> itself...
>>
>> If you believe that is the case, please try with CentOS 7.4 (can confirm
>> works fine) and/or CentOS 7.5.
>>
>> Best,
>> Andrija
>>
>>
>>
>> On Wed, 22 May 2019 at 09:04, Eric Lee Green <eric.lee.green@gmail.com>
>> wrote:
>>
>>> On 5/21/2019 11:10 PM, Thomas Joseph wrote:
>>>> Hello Eric,
>>>>
>>>>      If the router version is displayed as UNKNOWN in the portal, it
>>> would be
>>>> a connectivity issue. Check your connections related to firewall rules
>>>> between the ACP management hosts, hypervisor and VR.  Is your VR
>>> management
>>>> network setup correctly?
>>> Communications between the hosts is working fine and I have not changed
>>> the VR management network between the running 4.9 and the not-running
>>> 4.11. FIrewall rules appear to be the defaults set up by Cloudstack and
>>> should properly forward VR traffic.
>>>
>>> More tomorrow when I have some sleep since it is now midnight here in
>>> the SF Bay area.
>>>
>>>
>>>> On Wed, 22 May 2019, 6:10 am Eric Lee Green, <eric.lee.green@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks for the response, sorry if I sound frustrated, but this is
>>>>> supposed to be a simple easy process and it's been horrible all the way
>>>>> through. 4.11.1 failed so I had to downgrade back to 4.9, and now 4.11.2
>>>>> has failed to upgrade too. I've thus far spent around 16 hours of my
>>>>> time on what should have taken an hour at most. I'm frustrated and
>>> bummed.
>>>>> [root@cloud2 ~]# rpm -q centos-release
>>>>> centos-release-7-6.1810.2.el7.centos.x86_64
>>>>> [root@cloud2 ~]# rpm -q libvirt
>>>>> libvirt-4.5.0-10.el7_6.9.x86_64
>>>>> [root@cloud1 ~]# rpm -qa | grep kvm
>>>>> qemu-kvm-ev-2.12.0-18.el7_6.5.1.x86_64
>>>>> qemu-kvm-common-ev-2.12.0-18.el7_6.5.1.x86_64
>>>>> [root@cloud2 ~]# cat /proc/version
>>>>> Linux version 3.10.0-862.6.3.el7.x86_64
>>>>> (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat
>>>>> 4.8.5-28) (GCC) ) #1 SMP Tue Jun 26 16:32:21 UTC 2018
>>>>> [root@cloud2 ~]# cat /proc/cpuinfo
>>>>> ...
>>>>> processor       : 23
>>>>> vendor_id       : GenuineIntel
>>>>> cpu family      : 6
>>>>> model           : 44
>>>>> model name      : Intel(R) Xeon(R) CPU           X5675  @ 3.07GHz
>>>>> stepping        : 2
>>>>> microcode       : 0x1f
>>>>> cpu MHz         : 3068.000
>>>>> cache size      : 12288 KB
>>>>> physical id     : 1
>>>>> siblings        : 12
>>>>> core id         : 10
>>>>> cpu cores       : 6
>>>>> apicid          : 53
>>>>> initial apicid  : 53
>>>>> fpu             : yes
>>>>> fpu_exception   : yes
>>>>> cpuid level     : 11
>>>>> wp              : yes
>>>>> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>>>>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
>>>>> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
>>>>> nopl xtopology nonstop_tsc aperfmperf eagerfpu pni dtes64 monitor ds_cpl
>>>>> vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt
>>>>> lahf_lm epb ssbd ibrs ibpb tpr_shadow vnmi flexpriority ept vpid dtherm
>>>>> ida arat
>>>>> bogomips        : 6133.21
>>>>> clflush size    : 64
>>>>> cache_alignment : 64
>>>>> address sizes   : 40 bits physical, 48 bits virtual
>>>>> [root@cloud1 ~]# free
>>>>>                  total        used        free      shared buff/cache
>>>>> available
>>>>> Mem:      123596388    79795792    34047696      632116 9752900
>>> 39999332
>>>>> Swap:       4194300     1956824     2237476
>>>>> [root@cloud1 ~]# yum update
>>>>>
>>>>>
>>> =========================================================================================================================================================================
>>>>>     Package Arch Version Repository                             Size
>>>>>
>>>>>
>>> =========================================================================================================================================================================
>>>>> Updating:
>>>>>     cloudstack-agent x86_64 4.11.2.0-1.el7.centos
>>>>> cloudstack411                          47 M
>>>>>     cloudstack-common x86_64 4.11.2.0-1.el7.centos
>>>>> cloudstack411                          58 M
>>>>>     cloudstack-management x86_64 4.11.2.0-1.el7.centos
>>>>> cloudstack411                          82 M
>>>>>
>>>>> Three compute servers running the above processor averaging 128gb of
>>>>> memory apiece, two data servers running NFS for primary and secondary
>>>>> storage. Running the 4.11.2 community RPM's above.
>>>>>
>>>>> Yes, I did register the 4.11.2 systemvmtemplate before updating. The
>>>>> routers in fact started with the template, it said 4.11.2 when I opened
>>>>> up the console window and looked at the login prompt. But they never
got
>>>>> configured, when I logged in with root/password at the console prompt
>>>>> they had no networking set up and no configuration provided, the
>>>>> cloud-init output in /var/log was essentially blank.
>>>>>
>>>>> Do I recall correctly that there is an issue with a particular version
>>>>> of the hypervisor on the latest Centos 7? Any other information that
you
>>>>> need? I think I provided the complete log file for the cloud-init of
the
>>>>> router in another email...
>>>>>
>>>>> On 5/21/2019 9:38 PM, Rohit Yadav wrote:
>>>>>> Hi Eric,
>>>>>>
>>>>>> Can you describe your environment in detail such as management server
>>>>> host distro, hypervisor version, current CloudStack version, did you
>>>>> register the 4.11.2 systemvmtemplate beforw upgrading etc.
>>>>>> Regards.
>>>>>>
>>>>>> Regards,
>>>>>> Rohit Yadav
>>>>>>
>>>>>> ________________________________
>>>>>> From: Eric Lee Green <eric.lee.green@gmail.com>
>>>>>> Sent: Wednesday, May 22, 2019 6:21:16 AM
>>>>>> To: users@cloudstack.apache.org
>>>>>> Subject: Upgrade to Cloudstack 4.11.2 fails *AGAIN*
>>>>>>
>>>>>> You may remember me as the person who had to roll back to Cloudstack
>>>>>> 4.9.x because Cloudstack 4.11.1 wouldn't start any virtual machines
>>> once
>>>>>> I upgraded to it, claiming that there were inadequate resources even
>>>>>> though I had over 150 gigabytes of memory free in my cluster and
oodles
>>>>>> of CPU free (and a minimum of 40gb on each node, plenty to start
a
>>> 512mb
>>>>>> router VM). So now I'm trying to upgrade to Cloudstack 4.11.2 and
>>>>>> *again* it's misbehaving.
>>>>>>
>>>>>> The symptom is that my virtual routers when I log into their console
>>>>>> show 4.11.2 but when I look at them in the console they say 'Version:
>>>>>> UNKNOWN'. Also when I try to ssh into their guest IP address or link
>>>>>> local IP address it fails. And when I try to start up a virtual machine
>>>>>> that uses that virtual network, it says "Network unavailable', even
>>>>>> though the router for that network is showing up and running.
>>>>>>
>>>>>> Clearly something's broken in the virtual routers but I don't know
what
>>>>>> because I can't get into the router virtual machines. How do I get
the
>>>>>> console password to get into the router virtual machines? It's
>>> encrypted
>>>>>> in the database (duh), how do I decrypt it?
>>>>>>
>>>>>>
>>>>>>
>>>>>> rohit.yadav@shapeblue.com
>>>>>> www.shapeblue.com
>>>>>> Amadeus House, Floral Street, London  WC2E 9DPUK
>>>>>> @shapeblue
>>>>>>
>>>>>>
>>>>>>

Mime
View raw message