cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Lee Green <eric.lee.gr...@gmail.com>
Subject Re: Upgrade to Cloudstack 4.11.2 fails *AGAIN*
Date Wed, 22 May 2019 07:04:13 GMT

On 5/21/2019 11:10 PM, Thomas Joseph wrote:
> Hello Eric,
>
>     If the router version is displayed as UNKNOWN in the portal, it would be
> a connectivity issue. Check your connections related to firewall rules
> between the ACP management hosts, hypervisor and VR.  Is your VR management
> network setup correctly?

Communications between the hosts is working fine and I have not changed 
the VR management network between the running 4.9 and the not-running 
4.11. FIrewall rules appear to be the defaults set up by Cloudstack and 
should properly forward VR traffic.

More tomorrow when I have some sleep since it is now midnight here in 
the SF Bay area.


>
> On Wed, 22 May 2019, 6:10 am Eric Lee Green, <eric.lee.green@gmail.com>
> wrote:
>
>> Thanks for the response, sorry if I sound frustrated, but this is
>> supposed to be a simple easy process and it's been horrible all the way
>> through. 4.11.1 failed so I had to downgrade back to 4.9, and now 4.11.2
>> has failed to upgrade too. I've thus far spent around 16 hours of my
>> time on what should have taken an hour at most. I'm frustrated and bummed.
>>
>> [root@cloud2 ~]# rpm -q centos-release
>> centos-release-7-6.1810.2.el7.centos.x86_64
>> [root@cloud2 ~]# rpm -q libvirt
>> libvirt-4.5.0-10.el7_6.9.x86_64
>> [root@cloud1 ~]# rpm -qa | grep kvm
>> qemu-kvm-ev-2.12.0-18.el7_6.5.1.x86_64
>> qemu-kvm-common-ev-2.12.0-18.el7_6.5.1.x86_64
>> [root@cloud2 ~]# cat /proc/version
>> Linux version 3.10.0-862.6.3.el7.x86_64
>> (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat
>> 4.8.5-28) (GCC) ) #1 SMP Tue Jun 26 16:32:21 UTC 2018
>> [root@cloud2 ~]# cat /proc/cpuinfo
>> ...
>> processor       : 23
>> vendor_id       : GenuineIntel
>> cpu family      : 6
>> model           : 44
>> model name      : Intel(R) Xeon(R) CPU           X5675  @ 3.07GHz
>> stepping        : 2
>> microcode       : 0x1f
>> cpu MHz         : 3068.000
>> cache size      : 12288 KB
>> physical id     : 1
>> siblings        : 12
>> core id         : 10
>> cpu cores       : 6
>> apicid          : 53
>> initial apicid  : 53
>> fpu             : yes
>> fpu_exception   : yes
>> cpuid level     : 11
>> wp              : yes
>> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
>> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
>> nopl xtopology nonstop_tsc aperfmperf eagerfpu pni dtes64 monitor ds_cpl
>> vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt
>> lahf_lm epb ssbd ibrs ibpb tpr_shadow vnmi flexpriority ept vpid dtherm
>> ida arat
>> bogomips        : 6133.21
>> clflush size    : 64
>> cache_alignment : 64
>> address sizes   : 40 bits physical, 48 bits virtual
>> [root@cloud1 ~]# free
>>                 total        used        free      shared buff/cache
>> available
>> Mem:      123596388    79795792    34047696      632116 9752900    39999332
>> Swap:       4194300     1956824     2237476
>> [root@cloud1 ~]# yum update
>>
>> =========================================================================================================================================================================
>>    Package Arch Version Repository                             Size
>>
>> =========================================================================================================================================================================
>> Updating:
>>    cloudstack-agent x86_64 4.11.2.0-1.el7.centos
>> cloudstack411                          47 M
>>    cloudstack-common x86_64 4.11.2.0-1.el7.centos
>> cloudstack411                          58 M
>>    cloudstack-management x86_64 4.11.2.0-1.el7.centos
>> cloudstack411                          82 M
>>
>> Three compute servers running the above processor averaging 128gb of
>> memory apiece, two data servers running NFS for primary and secondary
>> storage. Running the 4.11.2 community RPM's above.
>>
>> Yes, I did register the 4.11.2 systemvmtemplate before updating. The
>> routers in fact started with the template, it said 4.11.2 when I opened
>> up the console window and looked at the login prompt. But they never got
>> configured, when I logged in with root/password at the console prompt
>> they had no networking set up and no configuration provided, the
>> cloud-init output in /var/log was essentially blank.
>>
>> Do I recall correctly that there is an issue with a particular version
>> of the hypervisor on the latest Centos 7? Any other information that you
>> need? I think I provided the complete log file for the cloud-init of the
>> router in another email...
>>
>> On 5/21/2019 9:38 PM, Rohit Yadav wrote:
>>> Hi Eric,
>>>
>>> Can you describe your environment in detail such as management server
>> host distro, hypervisor version, current CloudStack version, did you
>> register the 4.11.2 systemvmtemplate beforw upgrading etc.
>>> Regards.
>>>
>>> Regards,
>>> Rohit Yadav
>>>
>>> ________________________________
>>> From: Eric Lee Green <eric.lee.green@gmail.com>
>>> Sent: Wednesday, May 22, 2019 6:21:16 AM
>>> To: users@cloudstack.apache.org
>>> Subject: Upgrade to Cloudstack 4.11.2 fails *AGAIN*
>>>
>>> You may remember me as the person who had to roll back to Cloudstack
>>> 4.9.x because Cloudstack 4.11.1 wouldn't start any virtual machines once
>>> I upgraded to it, claiming that there were inadequate resources even
>>> though I had over 150 gigabytes of memory free in my cluster and oodles
>>> of CPU free (and a minimum of 40gb on each node, plenty to start a 512mb
>>> router VM). So now I'm trying to upgrade to Cloudstack 4.11.2 and
>>> *again* it's misbehaving.
>>>
>>> The symptom is that my virtual routers when I log into their console
>>> show 4.11.2 but when I look at them in the console they say 'Version:
>>> UNKNOWN'. Also when I try to ssh into their guest IP address or link
>>> local IP address it fails. And when I try to start up a virtual machine
>>> that uses that virtual network, it says "Network unavailable', even
>>> though the router for that network is showing up and running.
>>>
>>> Clearly something's broken in the virtual routers but I don't know what
>>> because I can't get into the router virtual machines. How do I get the
>>> console password to get into the router virtual machines? It's encrypted
>>> in the database (duh), how do I decrypt it?
>>>
>>>
>>>
>>> rohit.yadav@shapeblue.com
>>> www.shapeblue.com
>>> Amadeus House, Floral Street, London  WC2E 9DPUK
>>> @shapeblue
>>>
>>>
>>>

Mime
View raw message