cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Florent Paillot <florent.pail...@inria.fr>
Subject Re: Problem with ACS 4.11.2 and vCenter server
Date Wed, 17 Apr 2019 07:12:14 GMT
Some times ago, the cluster worked.

Here are the entries in the DB :

mysql> select id,name,uuid,guid,pod_id,data_center_id,hypervisor_type,allocation_state,engine_state
from cloud.cluster where id=16;
+----+--------------------------------+--------------------------------------+--------------------------------------+--------+----------------+-----------------+------------------+--------------+
| id | name                           | uuid                                 | guid      
                          | pod_id | data_center_id | hypervisor_type | allocation_state |
engine_state |
+----+--------------------------------+--------------------------------------+--------------------------------------+--------+----------------+-----------------+------------------+--------------+
| 16 | vcenter1/datacenter1/cluster1  | a95659a5-8918-438d-a3c9-214e8fcdcd97 | c74d97b0-1eae-357e-84aa-9d5bade97baf
|      1 |              1 | VMware          | Enabled          | Disabled     |
+----+--------------------------------+--------------------------------------+--------------------------------------+--------+----------------+-----------------+------------------+--------------+


mysql> select * from cloud.cluster_details where cluster_id="16";
+------+------------+----------------------------+---------------------------------------------------------+
| id   | cluster_id | name                       | value                                 
                 |
+------+------------+----------------------------+---------------------------------------------------------+
| 1048 |         16 | memoryOvercommitRatio      | 1.0                                   
                 |
| 1049 |         16 | guestvswitchtype           | vmwaresvs                             
                 |
| 1050 |         16 | username                   | csadmin@vsphere.local                 
                 |
| 1051 |         16 | cpuOvercommitRatio         | 1.0                                   
                 |
| 1052 |         16 | password                   | ANON	                                 
                 |
| 1053 |         16 | guestvswitchname           | vSwitch0                              
                 |
| 1055 |         16 | url                        | http://vcenter1/datacenter1/cluster1  
                 |
| 1063 |         16 | NativeHA                   | false                                 
                 |
| 1065 |         16 | outOfBandManagementEnabled | false                                 
                 |
| 1066 |         16 | resourceHAEnabled          | true                                  
                 |
+------+------------+----------------------------+---------------------------------------------------------+


mysql> select * from cloud.vmware_data_center where id=4;
+----+--------------------------------------+-------------------+----------------------------------+----------------+-----------------------+----------------------------------+
| id | uuid                                 | name              | guid                   
         | vcenter_host   | username              | password                         |
+----+--------------------------------------+-------------------+----------------------------------+----------------+-----------------------+----------------------------------+
|  4 | c39fe7c5-3478-43fa-aed1-38d40072279d | datacenter1       | datacenter1@vcenter    
         | vcenter1       | csadmin@vsphere.local | ANON                             |
+----+--------------------------------------+-------------------+----------------------------------+----------------+-----------------------+----------------------------------+


mysql> SELECT id,name,status,guid FROM cloud.host WHERE hypervisor_type = 'VMware' AND
status != 'Removed';
+----+-------+--------+------------------------------------+
| id | name  | status | guid                               |
+----+-------+--------+------------------------------------+
| 65 | esxi1 | Up     | HostSystem:host-973@vcenter1       |
+----+-------+--------+------------------------------------+



Host 65 is the ESXi hypervisor.


Is the log bellow from a version previous to 4.11 ?
Florent

----- Mail original -----
> De: "Sergey Levitskiy" <serg38l@hotmail.com>
> À: "users" <users@cloudstack.apache.org>
> Envoyé: Mardi 16 Avril 2019 22:59:46
> Objet: Re: Problem with ACS 4.11.2 and vCenter server

> It doesn’t look you have properly added VMware cluster & respective storage. Can
> you try again and post management server log which should have entries similar
> to below after you added cluster in UI. First you need to add VMware DC and
> then cluster.
> BTW you also have issues with one of your KVM hosts id=65.
> 
> 
> 2017-10-31 12:17:37,116 DEBUG [c.c.a.ApiServlet]
> (catalina-exec-22:ctx-f2dc50b2) (logid:799ac7c1) ===START===
> 172.17.101.246 -- POST  command=addVmwareDc&response=json
> 2017-10-31 12:17:37,132 INFO  [c.c.h.v.m.VmwareManagerImpl]
> (catalina-exec-22:ctx-f2dc50b2 ctx-a324e697) (logid:799ac7c1) Ignoring API
> call addVmwareDc, because VMware DC 172.17.101.246/WeDatacenter is already
> associated with specified zone with id 1
> 2017-10-31 12:17:37,133 DEBUG [c.c.a.ApiServlet]
> (catalina-exec-22:ctx-f2dc50b2 ctx-a324e697) (logid:799ac7c1) ===END===
> 172.17.101.246 -- POST  command=addVmwareDc&response=json
> 2017-10-31 12:17:37,155 DEBUG [c.c.a.ApiServlet]
> (catalina-exec-20:ctx-a26bd8ef) (logid:272077ae) ===START===
> 172.17.101.246 -- GET
> command=addCluster&zoneId=f538e730-7431-4c65-bf7d-eccc154e8b48&hypervisor=VMware&clustertype=ExternalManaged&podId=6262dca1-59df-4a32-b80a-52435f2ec4f7&username=administrator%40vsphere.local&url=http%3A%2F%2F172.17.101.246%2FWeDatacenter%2FWeCluster&clustername=172.17.101.246%2FWeDatacenter%2FWeCluster&response=json&_=1509432458636
> 2017-10-31 12:17:37,225 INFO  [c.c.h.v.VmwareServerDiscoverer]
> (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) Discover
> host. dc: 1, pod: 1, cluster: 7, uri host: 172.17.101.246
> 2017-10-31 12:17:37,313 INFO  [c.c.h.v.VmwareServerDiscoverer]
> (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) Detected
> private network label : DSwitch,,vmwaresvs
> 2017-10-31 12:17:37,313 DEBUG [c.c.h.v.r.VmwareContextFactory]
> (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) initialize
> VmwareContext. url: https://172.17.101.246/sdk/vimService, username:
> administrator@vsphere.local, password: W*********
> 2017-10-31 12:17:37,535 INFO  [c.c.h.v.u.VmwareContext]
> (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) New
> VmwareContext object, current outstanding count: 1
> 2017-10-31 12:17:37,662 INFO  [c.c.h.v.m.VmwareManagerImpl]
> (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) Preparing
> network on host com.cloud.hypervisor.vmware.util.VmwareContext@53bf4bc2 for
> DSwitch,,vmwaresvs
> 
> 
> 
> 
> 
> 
>On 4/16/19, 11:48 AM, "Florent Paillot" <florent.paillot@inria.fr> wrote:
> 
>    This is the full log :
>    https://filesender.renater.fr/?s=download&token=6be77301-3893-46fa-912c-3912086cae5f
>    
>    Florent
>    
>    ----- Mail original -----
>    > De: "Sergey Levitskiy" <serg38l@hotmail.com>
>    > À: "users" <users@cloudstack.apache.org>
>    > Envoyé: Mardi 16 Avril 2019 17:27:37
>    > Objet: Re: Problem with ACS 4.11.2 and vCenter server
>    
>    > Please post a full management-server.log without parsing.
>    > 
>    >On 4/16/19, 7:49 AM, "Florent Paillot" <florent.paillot@inria.fr> wrote:
>    > 
>    >    I did this on purpose to limit the size of the log.
>    >    
>    >    I let the controller run for more than 30mins. Can't find any revelant error...
>    >    :(
>    >    I put some grep results here : https://pad.inria.fr/p/np_IvcpeIGM4BncZEM1
>    >    
>    >    On the production controller (4.9.3), if we have connectivity problem with
the
>    >    vCenter, I see log with the "com.cloud.hypervisor.vmware.*" context.
>    >    Is there something to change in the log4j config to see this logs in 4.11
?
>    >    
>    >    The vim25.jar is the right one, after unziping
>    >    "/usr/share/cloudstack-management/lib/cloudstack-4.11.2.0.jar" in
>    >    com/vmware/vim25/vim.wsdl, I see " <soap:operation soapAction="urn:vim25/6.7"
>    >    style="document" />".
>    >    
>    >    
>    >    Florent
>    >    
>    >    ----- Mail original -----
>    >    > De: "Suresh Kumar Anaparti" <sureshkumar.anaparti@gmail.com>
>    >    > À: "users" <users@cloudstack.apache.org>
>    >    > Envoyé: Mardi 16 Avril 2019 14:22:28
>    >    > Objet: Re: Problem with ACS 4.11.2 and vCenter server
>    >    
>    >    > Florent, The log doesn't have any CS to vCenter connectivity details.
The
>    >    > CS management server was started and stopped in 2 mins. It needs some
time
>    >    > to load modules and connect to the hosts and storages. Wait for some
more
>    >    > time for the connectivity details in the log.
>    >    > 
>    >    > You can install vim25.jar for 6.7 version and give a try.
>    >    > 
>    >    > - Suresh
>    >    > 
>    >    > On Tue, Apr 16, 2019 at 5:34 PM Florent Paillot <florent.paillot@inria.fr>
>    >    > wrote:
>    >    > 
>    >    >> Here is the log : https://pad.inria.fr/p/np_IvcpeIGM4BncZEM1
>    >    >> It's a complete 3 minutes start/run/stop.
>    >    >>
>    >    >> Thanks for the info about "cs.vmware.api.version*". For 4.11.2, it
seems
>    >    >> that vSphere 6.7 is the target :
>    >    >> pom.xml:    <cs.vmware.api.version>6.7</cs.vmware.api.version>
>    >    >>
>    >    >>
>    >    >> Florent
>    >    >>
>    >    >> ----- Mail original -----
>    >    >> > De: "Suresh Kumar Anaparti" <sureshkumar.anaparti@gmail.com>
>    >    >> > À: "users" <users@cloudstack.apache.org>
>    >    >> > Envoyé: Mardi 16 Avril 2019 13:05:20
>    >    >> > Objet: Re: Problem with ACS 4.11.2 and vCenter server
>    >    >>
>    >    >> > Hi Florent,
>    >    >> >
>    >    >> > Log is missing. I think attachments are ignored. Either provide
a link to
>    >    >> > access the log or paste the logs in email (few logs of the issue).
>    >    >> >
>    >    >> > Irrespective of the vCenter version (hard coded in the build
param
>    >    >> > *cs.vmware.api.version*), CS would try to connect to the vCenter
server.
>    >    >> >
>    >    >> > - Suresh
>    >    >> >
>    >    >> >
>    >    >> > On Tue, Apr 16, 2019 at 2:44 PM Florent Paillot <
>    >    >> florent.paillot@inria.fr>
>    >    >> > wrote:
>    >    >> >
>    >    >> >> Thanks for the answers,
>    >    >> >>
>    >    >> >> Paul : I know that the vCenter 6.7 is not officialy supported
but my
>    >    >> >> colleague reported that it worked with 4.9.3 .
>    >    >> >> When you say "cloudstack won't recognise it", is it hard
coded ? If the
>    >    >> >> the vCenter auth API doesn't change between 6.5 and 6.7
it should work
>    >    >> or
>    >    >> >> at least try to connect to the vCenter.
>    >    >> >>
>    >    >> >> I'm agree with you Suresh, even if CS can't communicate
with the vCenter
>    >    >> >> we should see something in the logs or in the packet capture.
>    >    >> >>
>    >    >> >> I am able to ping and telnet from the CS controller to the
vCenter on
>    >    >> port
>    >    >> >> 443 (and I see the packet with tcpdump).
>    >    >> >>
>    >    >> >> The log is in attachement.
>    >    >> >>
>    >    >> >> Florent
>    >    >> >>
>    >    >> >>
>    >    >> >> ----- Mail original -----
>    >    >> >> > De: "Suresh Kumar Anaparti" <sureshkumar.anaparti@gmail.com>
>    >    >> >> > À: "users" <users@cloudstack.apache.org>
>    >    >> >> > Envoyé: Lundi 15 Avril 2019 17:54:00
>    >    >> >> > Objet: Re: Problem with ACS 4.11.2 and vCenter server
>    >    >> >>
>    >    >> >> > Hi Florent, What was the CS behavior & vCenter/ESXi
hosts connectivity
>    >    >> >> > status before upgrade to 4.11.2 from 4.9.3? Have you
upgraded vCenter
>    >    >> as
>    >    >> >> > well? Are you able to ping vCenter from CS management
server?
>    >    >> >> >
>    >    >> >> > Even vCenter 6.7 is not recognised, CS would try to
connect with the
>    >    >> >> > details specified, which is logged. Have you enabled
debug logs?
>    >    >> >> >
>    >    >> >> > - Suresh
>    >    >> >> >
>    >    >> >> > On Mon, Apr 15, 2019 at 7:38 PM Paul Angus <paul.angus@shapeblue.com>
>    >    >> >> wrote:
>    >    >> >> >
>    >    >> >> >> Ah. vCenter 6.7 isn't supported yet, cloudstack
won't recognise it.
>    >    >> >> >>
>    >    >> >> >> paul.angus@shapeblue.com
>    >    >> >> >> www.shapeblue.com
>    >    >> >> >> Amadeus House, Floral Street, London  WC2E 9DPUK
>    >    >> >> >> @shapeblue
>    >    >> >> >>
>    >    >> >> >>
>    >    >> >> >>
>    >    >> >> >>
>    >    >> >> >> -----Original Message-----
>    >    >> >> >> From: Florent Paillot <florent.paillot@inria.fr>
>    >    >> >> >> Sent: 15 April 2019 14:26
>    >    >> >> >> To: users <users@cloudstack.apache.org>
>    >    >> >> >> Subject: Re: Problem with ACS 4.11.2 and vCenter
server
>    >    >> >> >>
>    >    >> >> >> Hi Suresh,
>    >    >> >> >> The vCenter is a 6.7u2 (last build).
>    >    >> >> >>
>    >    >> >> >> There is no issue log, that the problem... I don't
see any error
>    >    >> message
>    >    >> >> >> concerning a failed connection to the vCenter and
CS does not seems
>    >    >> to
>    >    >> >> send
>    >    >> >> >> any packet to the vCenter.
>    >    >> >> >>
>    >    >> >> >> Florent
>    >    >> >> >>
>    >    >> >> >>
>    >    >> >> >> ----- Mail original -----
>    >    >> >> >> > De: "Suresh Kumar Anaparti" <sureshkumar.anaparti@gmail.com>
>    >    >> >> >> > À: "users" <users@cloudstack.apache.org>
>    >    >> >> >> > Envoyé: Lundi 15 Avril 2019 15:10:44
>    >    >> >> >> > Objet: Re: Problem with ACS 4.11.2 and vCenter
server
>    >    >> >> >>
>    >    >> >> >> > Hi Florent,
>    >    >> >> >> >
>    >    >> >> >> > Can you share the vCenter version and build
details. Also, the
>    >    >> >> >> > cloudstack management server log with the
issue logs while
>    >    >> connecting
>    >    >> >> to
>    >    >> >> >> vCenter.
>    >    >> >> >> >
>    >    >> >> >> > - Suresh
>    >    >> >> >> >
>    >    >> >> >> > On Mon, Apr 15, 2019 at 6:24 PM Paul Angus
<
>    >    >> paul.angus@shapeblue.com>
>    >    >> >> >> wrote:
>    >    >> >> >> >
>    >    >> >> >> >> Assuming that you have updated the core
code to recognise and use
>    >    >> >> >> 'newer'
>    >    >> >> >> >> libraries/SDKs than the ones currently
in use (and thoroughly
>    >    >> tested
>    >    >> >> >> >> it) then, please do create a pull request.
 Otherwise, I
>    >    >> personally
>    >    >> >> >> >> would advise that you leave things 'as-is'
to get the most stable
>    >    >> >> >> experience.
>    >    >> >> >> >>
>    >    >> >> >> >> I don't *think* that you need to run
>    >    >> >> >> >>
>    >    >> >> >> >> mvn clean
>    >    >> >> >> >> mvn install -Dnoredist
>    >    >> >> >> >>
>    >    >> >> >> >> *After* installing the non-oss components.
>    >    >> >> >> >>
>    >    >> >> >> >> You might want to do a mvn clean before
you start.
>    >    >> >> >> >>
>    >    >> >> >> >> It's been a while since I've done it manually
now.
>    >    >> >> >> >>
>    >    >> >> >> >>
>    >    >> >> >> >>
>    >    >> >> >> >> paul.angus@shapeblue.com
>    >    >> >> >> >> www.shapeblue.com
>    >    >> >> >> >> Amadeus House, Floral Street, London 
WC2E 9DPUK @shapeblue
>    >    >> >> >> >>
>    >    >> >> >> >>
>    >    >> >> >> >>
>    >    >> >> >> >>
>    >    >> >> >> >> -----Original Message-----
>    >    >> >> >> >> From: Florent Paillot <florent.paillot@inria.fr>
>    >    >> >> >> >> Sent: 15 April 2019 13:17
>    >    >> >> >> >> To: users <users@cloudstack.apache.org>
>    >    >> >> >> >> Subject: Re: Problem with ACS 4.11.2 and
vCenter server
>    >    >> >> >> >>
>    >    >> >> >> >> Yes, this is the list of the files in
deps/ :
>    >    >> >> >> >> cloud-iControl.jar  cloud-netscaler-sdx.jar
 iControl-12.1.0.jar
>    >    >> >> >> >> install-non-oss.sh  manageontap.jar  vim25_51.jar
 vim25_55.jar
>    >    >> >> >> >> vim25_60.jar  vim25_65.jar  vim25_67.jar
>    >    >> >> >> >>
>    >    >> >> >> >>
>    >    >> >> >> >> I had to add the following lines to "install-non-oss.sh"
to get
>    >    >> the
>    >    >> >> >> build
>    >    >> >> >> >> pass :
>    >    >> >> >> >>
>    >    >> >> >> >> mvn install:install-file -Dfile=vim25_67.jar
>    >    >> >> >> >> -DgroupId=com.cloud.com.vmware -DartifactId=vmware-vim25
>    >    >> >> >> -Dversion=6.7
>    >    >> >> >> >>  -Dpackaging=jar
>    >    >> >> >> >> mvn install:install-file -Dfile=iControl-12.1.0.jar
>    >    >> >> >> >>  -DgroupId=com.cloud.com.f5     -DartifactId=icontrol
>    >    >> >> >> >> -Dversion=12.1  -Dpackaging=jar
>    >    >> >> >> >>
>    >    >> >> >> >> I can open a PR or an issue if it makes
sense.
>    >    >> >> >> >>
>    >    >> >> >> >> Florent
>    >    >> >> >> >>
>    >    >> >> >> >> --
>    >    >> >> >> >> Florent PAILLOT
>    >    >> >> >> >> SESI - Cellule architecture
>    >    >> >> >> >>
>    >    >> >> >> >> INRIA Bordeaux -Sud-Ouest
>    >    >> >> >> >> 05.24.57.41.25
>    >    >> >> >> >>
>    >    >> >> >> >> ----- Mail original -----
>    >    >> >> >> >> > De: "Paul Angus" <paul.angus@shapeblue.com>
>    >    >> >> >> >> > À: "users" <users@cloudstack.apache.org>
>    >    >> >> >> >> > Envoyé: Lundi 15 Avril 2019 14:10:21
>    >    >> >> >> >> > Objet: RE: Problem with ACS 4.11.2
and vCenter server
>    >    >> >> >> >>
>    >    >> >> >> >> > When packaging, did you install all
of the nonoss libraries
>    >    >> into
>    >    >> >> the
>    >    >> >> >> >> > build before packaging?  including
vmware-vim25-6.5.jar
>    >    >> >> >> >> >
>    >    >> >> >> >> > paul.angus@shapeblue.com
>    >    >> >> >> >> > www.shapeblue.com
>    >    >> >> >> >> > Amadeus House, Floral Street, London
 WC2E 9DPUK @shapeblue
>    >    >> >> >> >> >
>    >    >> >> >> >> >
>    >    >> >> >> >> >
>    >    >> >> >> >> >
>    >    >> >> >> >> > -----Original Message-----
>    >    >> >> >> >> > From: Dag Sonstebo <Dag.Sonstebo@shapeblue.com>
>    >    >> >> >> >> > Sent: 15 April 2019 12:43
>    >    >> >> >> >> > To: users@cloudstack.apache.org
>    >    >> >> >> >> > Subject: Re: Problem with ACS 4.11.2
and vCenter server
>    >    >> >> >> >> >
>    >    >> >> >> >> > Florent,
>    >    >> >> >> >> >
>    >    >> >> >> >> > Not aware of any issues but I'm no
packaging expert so will let
>    >    >> >> others
>    >    >> >> >> >> > respond to this.
>    >    >> >> >> >> >
>    >    >> >> >> >> > However - any reason you're not just
using the RPM repositories
>    >    >> >> rather
>    >    >> >> >> >> > than packaging yourself?
>    >    >> >> >> >> >
>    >    >> >> >> >> > Regards,
>    >    >> >> >> >> > Dag Sonstebo
>    >    >> >> >> >> > Cloud Architect
>    >    >> >> >> >> > ShapeBlue
>    >    >> >> >> >> >
>    >    >> >> >> >> >
>    >    >> >> >> >> >On 15/04/2019, 11:46, "Florent Paillot"
<
>    >    >> florent.paillot@inria.fr>
>    >    >> >> >> >> wrote:
>    >    >> >> >> >> >
>    >    >> >> >> >> >    Hi everyone,
>    >    >> >> >> >> >    We have a problem with our test
environment running ACS
>    >    >> 4.11.2
>    >    >> >> on
>    >    >> >> >> >> CentOS 7, not
>    >    >> >> >> >> >    communicating with a vCenter (upgraded
from 4.9.3 to 4.11.2
>    >    >> and
>    >    >> >> >> >> CentOS 6 o
>    >    >> >> >> >> >    CentOS 7). We have no problems
with our KVM host.
>    >    >> >> >> >> >    Our production is using 4.9.3
o, Centos 6 and everything is
>    >    >> OK.
>    >    >> >> >> >> >
>    >    >> >> >> >> >    I used tcpdump and see nothing
going out from the management
>    >    >> >> server
>    >    >> >> >> >> to the
>    >    >> >> >> >> >    vCenter or the ESXi host.
>    >    >> >> >> >> >
>    >    >> >> >> >> >    Could it be a package building
problem or a vmware module
>    >    >> >> loading?
>    >    >> >> >> >> >
>    >    >> >> >> >> >    I used the following commands
to build the packages :
>    >    >> >> >> >> >    install-non-oss.sh
>    >    >> >> >> >> >    mvn clean
>    >    >> >> >> >> >    mvn install -Dnoredist
>    >    >> >> >> >> >    package.sh -p noredist -d centos7
>    >    >> >> >> >> >
>    >    >> >> >> >> >
>    >    >> >> >> >> >    On the production server I see
a :
>    >    >> >> >> >> >
>    >    >> >> >> >>
>    >    >> >> >>
>    >    >> >>
>    >    >> /usr/share/cloudstack-management/webapps/client/WEB-INF/lib/vmware-vim25-6.0.jar
>    >    >> >> >> >> >    file.
>    >    >> >> >> >> >    I can't find it on the test environnement,
is it linked to
>    >    >> >> jetty ?
>    >    >> >> >> >> >
>    >    >> >> >> >> >    Any help is welcome :)
>    >    >> >> >> >> >    Regards,
>    >    >> >> >> >> >
>    >    >> >> >> >> >
>    >    >> >> >> >> >
>    >    >> >> >> >> >
>    >    >> >> >> >> >
>    >    >> >> >> >> > Dag.Sonstebo@shapeblue.com
>    >    >> >> >> >> > www.shapeblue.com
>     >     >> >> >> >> > Amadeus House, Floral Street, London
 WC2E 9DPUK @shapeblue

Mime
View raw message