cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrija Panic <andrija.pa...@gmail.com>
Subject Re: Upgrade to Cloudstack 4.11.2 fails *AGAIN*
Date Wed, 22 May 2019 14:56:34 GMT
Eric,

did you actually test this in production?

Andrija

On Wed, 22 May 2019 at 16:33, Eric Lee Green <eric.lee.green@gmail.com>
wrote:

> Okay. This makes sense.
>
> And people wonder why Amazon decided to make their own Linux rather than
> use Centos and why Ubuntu has seized huge market share from Red Hat in
> the past few years. SIGH.
>
> Downgrading my CentOS is not going to be easy. There were security
> patches in the latest CentOS that I am required to have. It would
> actually be easier to just switch to Ubuntu at this point since we're
> talking a complete re-install anyhow. SIGH. Thank you, Red Hat Software.
> I hope IBM fixes them, but I have my doubts.
>
> On 5/22/2019 1:05 AM, Andrija Panic wrote:
> > Hi Eric, all,
> >
> > I believe you might be hitting this one - issues on latest CentOS 7.6:
> > https://github.com/apache/cloudstack/pull/3333 due to changes in the OS
> > itself...
> >
> > If you believe that is the case, please try with CentOS 7.4 (can confirm
> > works fine) and/or CentOS 7.5.
> >
> > Best,
> > Andrija
> >
> >
> >
> > On Wed, 22 May 2019 at 09:04, Eric Lee Green <eric.lee.green@gmail.com>
> > wrote:
> >
> >> On 5/21/2019 11:10 PM, Thomas Joseph wrote:
> >>> Hello Eric,
> >>>
> >>>      If the router version is displayed as UNKNOWN in the portal, it
> >> would be
> >>> a connectivity issue. Check your connections related to firewall rules
> >>> between the ACP management hosts, hypervisor and VR.  Is your VR
> >> management
> >>> network setup correctly?
> >> Communications between the hosts is working fine and I have not changed
> >> the VR management network between the running 4.9 and the not-running
> >> 4.11. FIrewall rules appear to be the defaults set up by Cloudstack and
> >> should properly forward VR traffic.
> >>
> >> More tomorrow when I have some sleep since it is now midnight here in
> >> the SF Bay area.
> >>
> >>
> >>> On Wed, 22 May 2019, 6:10 am Eric Lee Green, <eric.lee.green@gmail.com
> >
> >>> wrote:
> >>>
> >>>> Thanks for the response, sorry if I sound frustrated, but this is
> >>>> supposed to be a simple easy process and it's been horrible all the
> way
> >>>> through. 4.11.1 failed so I had to downgrade back to 4.9, and now
> 4.11.2
> >>>> has failed to upgrade too. I've thus far spent around 16 hours of my
> >>>> time on what should have taken an hour at most. I'm frustrated and
> >> bummed.
> >>>> [root@cloud2 ~]# rpm -q centos-release
> >>>> centos-release-7-6.1810.2.el7.centos.x86_64
> >>>> [root@cloud2 ~]# rpm -q libvirt
> >>>> libvirt-4.5.0-10.el7_6.9.x86_64
> >>>> [root@cloud1 ~]# rpm -qa | grep kvm
> >>>> qemu-kvm-ev-2.12.0-18.el7_6.5.1.x86_64
> >>>> qemu-kvm-common-ev-2.12.0-18.el7_6.5.1.x86_64
> >>>> [root@cloud2 ~]# cat /proc/version
> >>>> Linux version 3.10.0-862.6.3.el7.x86_64
> >>>> (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red
> Hat
> >>>> 4.8.5-28) (GCC) ) #1 SMP Tue Jun 26 16:32:21 UTC 2018
> >>>> [root@cloud2 ~]# cat /proc/cpuinfo
> >>>> ...
> >>>> processor       : 23
> >>>> vendor_id       : GenuineIntel
> >>>> cpu family      : 6
> >>>> model           : 44
> >>>> model name      : Intel(R) Xeon(R) CPU           X5675  @ 3.07GHz
> >>>> stepping        : 2
> >>>> microcode       : 0x1f
> >>>> cpu MHz         : 3068.000
> >>>> cache size      : 12288 KB
> >>>> physical id     : 1
> >>>> siblings        : 12
> >>>> core id         : 10
> >>>> cpu cores       : 6
> >>>> apicid          : 53
> >>>> initial apicid  : 53
> >>>> fpu             : yes
> >>>> fpu_exception   : yes
> >>>> cpuid level     : 11
> >>>> wp              : yes
> >>>> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> >>>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
> >>>> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts
> rep_good
> >>>> nopl xtopology nonstop_tsc aperfmperf eagerfpu pni dtes64 monitor
> ds_cpl
> >>>> vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt
> >>>> lahf_lm epb ssbd ibrs ibpb tpr_shadow vnmi flexpriority ept vpid
> dtherm
> >>>> ida arat
> >>>> bogomips        : 6133.21
> >>>> clflush size    : 64
> >>>> cache_alignment : 64
> >>>> address sizes   : 40 bits physical, 48 bits virtual
> >>>> [root@cloud1 ~]# free
> >>>>                  total        used        free      shared buff/cache
> >>>> available
> >>>> Mem:      123596388    79795792    34047696      632116 9752900
> >> 39999332
> >>>> Swap:       4194300     1956824     2237476
> >>>> [root@cloud1 ~]# yum update
> >>>>
> >>>>
> >>
> =========================================================================================================================================================================
> >>>>     Package Arch Version Repository                             Size
> >>>>
> >>>>
> >>
> =========================================================================================================================================================================
> >>>> Updating:
> >>>>     cloudstack-agent x86_64 4.11.2.0-1.el7.centos
> >>>> cloudstack411                          47 M
> >>>>     cloudstack-common x86_64 4.11.2.0-1.el7.centos
> >>>> cloudstack411                          58 M
> >>>>     cloudstack-management x86_64 4.11.2.0-1.el7.centos
> >>>> cloudstack411                          82 M
> >>>>
> >>>> Three compute servers running the above processor averaging 128gb of
> >>>> memory apiece, two data servers running NFS for primary and secondary
> >>>> storage. Running the 4.11.2 community RPM's above.
> >>>>
> >>>> Yes, I did register the 4.11.2 systemvmtemplate before updating. The
> >>>> routers in fact started with the template, it said 4.11.2 when I
> opened
> >>>> up the console window and looked at the login prompt. But they never
> got
> >>>> configured, when I logged in with root/password at the console prompt
> >>>> they had no networking set up and no configuration provided, the
> >>>> cloud-init output in /var/log was essentially blank.
> >>>>
> >>>> Do I recall correctly that there is an issue with a particular version
> >>>> of the hypervisor on the latest Centos 7? Any other information that
> you
> >>>> need? I think I provided the complete log file for the cloud-init of
> the
> >>>> router in another email...
> >>>>
> >>>> On 5/21/2019 9:38 PM, Rohit Yadav wrote:
> >>>>> Hi Eric,
> >>>>>
> >>>>> Can you describe your environment in detail such as management server
> >>>> host distro, hypervisor version, current CloudStack version, did you
> >>>> register the 4.11.2 systemvmtemplate beforw upgrading etc.
> >>>>> Regards.
> >>>>>
> >>>>> Regards,
> >>>>> Rohit Yadav
> >>>>>
> >>>>> ________________________________
> >>>>> From: Eric Lee Green <eric.lee.green@gmail.com>
> >>>>> Sent: Wednesday, May 22, 2019 6:21:16 AM
> >>>>> To: users@cloudstack.apache.org
> >>>>> Subject: Upgrade to Cloudstack 4.11.2 fails *AGAIN*
> >>>>>
> >>>>> You may remember me as the person who had to roll back to Cloudstack
> >>>>> 4.9.x because Cloudstack 4.11.1 wouldn't start any virtual machines
> >> once
> >>>>> I upgraded to it, claiming that there were inadequate resources
even
> >>>>> though I had over 150 gigabytes of memory free in my cluster and
> oodles
> >>>>> of CPU free (and a minimum of 40gb on each node, plenty to start
a
> >> 512mb
> >>>>> router VM). So now I'm trying to upgrade to Cloudstack 4.11.2 and
> >>>>> *again* it's misbehaving.
> >>>>>
> >>>>> The symptom is that my virtual routers when I log into their console
> >>>>> show 4.11.2 but when I look at them in the console they say 'Version:
> >>>>> UNKNOWN'. Also when I try to ssh into their guest IP address or
link
> >>>>> local IP address it fails. And when I try to start up a virtual
> machine
> >>>>> that uses that virtual network, it says "Network unavailable', even
> >>>>> though the router for that network is showing up and running.
> >>>>>
> >>>>> Clearly something's broken in the virtual routers but I don't know
> what
> >>>>> because I can't get into the router virtual machines. How do I get
> the
> >>>>> console password to get into the router virtual machines? It's
> >> encrypted
> >>>>> in the database (duh), how do I decrypt it?
> >>>>>
> >>>>>
> >>>>>
> >>>>> rohit.yadav@shapeblue.com
> >>>>> www.shapeblue.com
> >>>>> Amadeus House, Floral Street, London  WC2E 9DPUK
> >>>>> @shapeblue
> >>>>>
> >>>>>
> >>>>>
> >
>


-- 

Andrija Panić

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message