cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam <adam.scarce...@gmail.com>
Subject Re: Management Server Startup Issue
Date Tue, 15 Jul 2014 20:24:35 GMT
I do not use awsapi either, and have never turned it on. If I look in the
database I can see that those features are turned off too:

SELECT * FROM cloud.configuration where name = 'enable.ec2.api' or name =
'enable.s3.api';

# category, instance, component, name, value, description, default_value,
updated, scope, is_dynamic
'Advanced', 'DEFAULT', 'management-server', 'enable.ec2.api', 'false',
'enable EC2 API on CloudStack', 'false', NULL, NULL, '0'
'Advanced', 'DEFAULT', 'management-server', 'enable.s3.api', 'false',
'enable Amazon S3 API on CloudStack', 'false', NULL, NULL, '0'


I found this though: https://issues.apache.org/jira/browse/CLOUDSTACK-2758

Not sure if it's related.

I tried deleted everything from both the work and temp dirs, but it had no
affect.

:(

Best Regards,



Adam Scarcella


On Tue, Jul 15, 2014 at 12:22 PM, Rafael Weingartner <
rafaelweingartner@gmail.com> wrote:

> Have you tried to clean the directories work and temp of the tomcat
> installation that the CS is using?
>
> rm -rf /usr/share/cloudstack-management/temp/*
>
> rm -rf /usr/share/cloudstack-management/work/*
>
>
> In my CS installation I do not even have the awsapi files.
>
> I have just noticed that now, it has a soft link pointing to
> ../../cloudstack-bridge/webapps/awsapi, However, that folder does not
> exist.
>
> Are you using awsapi?
>
> I do not use it, so that is not a problem.
>
>
> On Tue, Jul 15, 2014 at 1:05 PM, Adam <adam.scarcella@gmail.com> wrote:
>
> > OK, so I have this issue HTTP Status 404
> >
> > [image: Inline image 1]
> >
> >
> > ...except it must be different in some way because I've gone through this
> > entire thread and even though I have the same original error mentioned
> and
> > the same behavior, I do not have the same "smoking gun" so to speak in
> the
> > localhost.log where my logging type was STATEMENT instead of  MIXED.
> >
> > [root@cs-east-prod1 logs]# cat /etc/my.cnf
> > [mysqld]
> > datadir=/var/lib/mysql
> > sockeet=/var/lib/mysql/mysql.sock
> > user=mysql
> > # Disabling symbolic-links is recommended to prevent assorted security
> > risks
> > symbolic-links=0
> >
> >
> > #
> > # BEGIN CloudStack 4.3 Requirements
> > #
> > innodb_rollback_on_timeout=1
> > innodb_lock_wait_timeout=600
> > max_connections=350
> > log-bin=mysql-bin
> > binlog-format = 'ROW'
> > #
> > # END CloudStack 4.3 Requirements
> > #
> >
> > [mysqld_safe]
> > log-error=/var/log/mysqld.log
> > pid-file=/var/run/mysqld/mysqld.pid
> > [root@cs-east-prod1 logs]#
> >
> >
> >
> > I upgraded from 4.2.1 to 4.3 (CentOS 6.5 x86_64, KVM) and ever since I've
> > not been able to start the cloudstack-management ui. I see these errors
> in
> > the catalina.out log:
> >
> > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer
> deploy
> > SEVERE: The addressing-1.5.4.mar module, which is not valid, caused The
> > module.xml file cannot be found for the module:
> >
> /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar
> > org.apache.axis2.deployment.DeploymentException: The module.xml file
> > cannot be found for the module:
> >
> /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar
> >         at
> >
> org.apache.axis2.deployment.repository.util.ArchiveReader.readModuleArchive(ArchiveReader.java:529)
> >         at
> > org.apache.axis2.deployment.ModuleDeployer.deploy(ModuleDeployer.java:75)
> >         at
> >
> org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)
> >         at
> >
> org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:661)
> >         at
> >
> org.apache.axis2.deployment.RepositoryListener.init(RepositoryListener.java:251)
> >         at
> >
> org.apache.axis2.deployment.RepositoryListener.init2(RepositoryListener.java:68)
> >         at
> >
> org.apache.axis2.deployment.RepositoryListener.<init>(RepositoryListener.java:63)
> >         at
> >
> org.apache.axis2.deployment.DeploymentEngine.loadRepository(DeploymentEngine.java:147)
> >         at
> >
> org.apache.axis2.deployment.WarBasedAxisConfigurator.getAxisConfiguration(WarBasedAxisConfigurator.java:229)
> >         at
> >
> org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContext(ConfigurationContextFactory.java:64)
> >         at
> >
> org.apache.axis2.transport.http.AxisServlet.initConfigContext(AxisServlet.java:567)
> >         at
> > org.apache.axis2.transport.http.AxisServlet.init(AxisServlet.java:452)
> >         at
> >
> org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173)
> >         at
> > org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:993)
> >         at
> >
> org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4187)
> >         at
> > org.apache.catalina.core.StandardContext.start(StandardContext.java:4496)
> >         at
> >
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> >         at
> > org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> >         at
> > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> >         at
> >
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> >         at
> >
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> >         at
> > org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> >         at
> > org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> >         at
> >
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> >         at
> >
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
> >         at
> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> >         at
> > org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> >         at
> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> >         at
> > org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> >         at
> > org.apache.catalina.core.StandardService.start(StandardService.java:516)
> >         at
> > org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> >         at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:606)
> >         at
> org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> >         at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> > Caused by: org.apache.axis2.deployment.DeploymentException: The
> module.xml
> > file cannot be found for the module:
> >
> /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar
> >         at
> >
> org.apache.axis2.deployment.repository.util.ArchiveReader.readModuleArchive(ArchiveReader.java:523)
> >         ... 37 more
> >
> > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer
> deploy
> > INFO: Deploying module: addressing-1.5.6 -
> >
> file:/usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.6.mar
> > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer
> deploy
> > INFO: Deploying module: jaxws-1.5.6 -
> >
> file:/usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/axis2-jaxws-mar-1.5.6.mar
> > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer
> deploy
> > INFO: Deploying module: mtompolicy-1.5.6 -
> >
> file:/usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/mtompolicy-1.5.6.mar
> > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer
> deploy
> > INFO: Deploying module: addressing-1.5.6 -
> >
> file:/usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/lib/axis2-1.5.6.jar
> > Jul 15, 2014 10:13:57 AM org.apache.catalina.loader.WebappClassLoader
> > loadClass
> > INFO: Illegal access: this web application instance has been stopped
> > already.  Could not load
> > org.apache.cloudstack.managed.context.ManagedContextTimerTask$1.  The
> > eventual following stack trace is caused by an error thrown for debugging
> > purposes as well as to attempt to terminate the thread which caused the
> > illegal access, and has no functional impact.
> > java.lang.IllegalStateException
> >         at
> >
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1369)
> >         at
> >
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1329)
> >         at
> >
> org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:27)
> >         at java.util.TimerThread.mainLoop(Timer.java:555)
> >         at java.util.TimerThread.run(Timer.java:505)
> >
> > Exception in thread "Timer-2" java.lang.NoClassDefFoundError:
> > org/apache/cloudstack/managed/context/ManagedContextTimerTask$1
> >         at
> >
> org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:27)
> >         at java.util.TimerThread.mainLoop(Timer.java:555)
> >         at java.util.TimerThread.run(Timer.java:505)
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.cloudstack.managed.context.ManagedContextTimerTask$1
> >         at
> >
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1483)
> >         at
> >
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1329)
> >         ... 3 more
> >
> >
> > I have absolutely no errors of any kind in the management-server.log.
> >
> > [root@cs-east-prod1 logs]# grep -i -E
> > 'exception|unable|fail|invalid|leak|warn|error' ./management-server.log
> > [root@cs-east-prod1 logs]#
> >
> >
> > The only thing I see in the access.log is:
> >
> > [root@cs-east-prod1 logs]# cat access_log.2014-07-15.txt
> > 10.97.34.115 - - [15/Jul/2014:10:27:15 -0500] "GET /client/ HTTP/1.1" 404
> > 952
> > [root@cs-east-prod1 logs]#
> >
> >
> > ... and my localhost.log is entirely empty.
> >
> >
> > [root@cs-east-prod1 logs]# du -sh localhost.2014-07-15.log
> > 0       localhost.2014-07-15.log
> >
> >
> > And even more bizzare is the fact that I have another CloudStack 4.3
> > server that is running on XenServer and it's perfectly fine. It was not
> an
> > upgrade though, it was a clean install of 4.3. I've compared all the
> files
> > associated with the addressing-1.5.4.mar but could not find anything
> > significant.
> >
> > Here is the output from my upgraded server (cs-east-prod1: broken with
> 404)
> >
> > [root@cs-east-prod1 modules]# pwd
> > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules
> > [root@cs-east-prod1 modules]# md5sum addressing-1.5.4.mar
> > 1ef211c1a042e7d4b9b7e5f914222929  addressing-1.5.4.mar
> > [root@cs-east-prod1 modules]# cat modules.list
> > soapmonitor-1.5.6.mar
> > addressing-1.5.6.mar
> > mtompolicy-1.5.6.mar
> > mex-1.5.6.mar
> > ping-1.5.6.mar
> > axis2-jaxws-mar-1.5.6.mar
> > scripting-1.5.6.mar
> > [root@cs-east-prod1 modules]# ls -l
> > total 340
> > -rwxrwxrwx 1 cloud cloud 126981 Mar 25 11:01 addressing-1.5.4.mar
> > -rwxrwxrwx 1 cloud cloud  34518 Mar 25 11:01 addressing-1.5.6.mar
> > -rwxrwxrwx 1 cloud cloud  14363 Mar 25 11:01 axis2-jaxws-mar-1.5.6.mar
> > -rwxrwxrwx 1 cloud cloud  38620 Mar 25 11:01 mex-1.5.6.mar
> > -rwxrwxrwx 1 cloud cloud    139 Mar 25 11:01 modules.list
> > -rwxrwxrwx 1 cloud cloud  18682 Mar 25 11:01 mtompolicy-1.5.6.mar
> > -rwxrwxrwx 1 cloud cloud  17647 Mar 25 11:01 ping-1.5.6.mar
> > -rwxrwxrwx 1 cloud cloud   9721 Mar 25 11:01 rahas-1.5.1.mar
> > -rwxrwxrwx 1 cloud cloud   9612 Mar 25 11:01 rampart-1.5.1.mar
> > -rwxrwxrwx 1 cloud cloud  33779 Mar 25 11:01 scripting-1.5.6.mar
> > -rwxrwxrwx 1 cloud cloud  15650 Mar 25 11:01 soapmonitor-1.5.6.mar
> > [root@cs-east-prod1 modules]# cd /
> > [root@cs-east-prod1 /]# find . -name "module.xml"
> >
> >
> ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml
> >
> >
> ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml
> > [root@cs-east-prod1 /]# md5sum
> >
> ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml
> > baf5f63f5dd57484e9a88be6689939fc
> >
>  ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml
> > [root@cs-east-prod1 /]# md5sum
> >
> ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml
> > 913b4561258980f7b1b4c7cb3de261ba
> >
>  ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml
> > [root@cs-east-prod1 /]#
> >
> >
> > And here is the output from the exact same commands on my 4.3 server
> > that's working fine (na-nyc-bos-cs-poc2):
> >
> > [root@na-nyc-bos-cs-poc2 modules]# pwd
> > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules
> > [root@na-nyc-bos-cs-poc2 modules]# md5sum addressing-1.5.4.mar
> > 1ef211c1a042e7d4b9b7e5f914222929  addressing-1.5.4.mar
> > [root@na-nyc-bos-cs-poc2 modules]# cat modules.list
> > soapmonitor-1.5.6.mar
> > addressing-1.5.6.mar
> > mtompolicy-1.5.6.mar
> > mex-1.5.6.mar
> > ping-1.5.6.mar
> > axis2-jaxws-mar-1.5.6.mar
> > scripting-1.5.6.mar
> > [root@na-nyc-bos-cs-poc2 modules]# ls -l
> > total 340
> > -rw-r--r--. 1 cloud cloud 126981 Mar 25 11:01 addressing-1.5.4.mar
> > -rw-r--r--. 1 cloud cloud  34518 Mar 25 11:01 addressing-1.5.6.mar
> > -rw-r--r--. 1 cloud cloud  14363 Mar 25 11:01 axis2-jaxws-mar-1.5.6.mar
> > -rw-r--r--. 1 cloud cloud  38620 Mar 25 11:01 mex-1.5.6.mar
> > -rw-r--r--. 1 cloud cloud    139 Mar 25 11:01 modules.list
> > -rw-r--r--. 1 cloud cloud  18682 Mar 25 11:01 mtompolicy-1.5.6.mar
> > -rw-r--r--. 1 cloud cloud  17647 Mar 25 11:01 ping-1.5.6.mar
> > -rw-r--r--. 1 cloud cloud   9721 Mar 25 11:01 rahas-1.5.1.mar
> > -rw-r--r--. 1 cloud cloud   9612 Mar 25 11:01 rampart-1.5.1.mar
> > -rw-r--r--. 1 cloud cloud  33779 Mar 25 11:01 scripting-1.5.6.mar
> > -rw-r--r--. 1 cloud cloud  15650 Mar 25 11:01 soapmonitor-1.5.6.mar
> > [root@na-nyc-bos-cs-poc2 modules]# cd /
> > [root@na-nyc-bos-cs-poc2 /]# find . -name "module.xml"
> >
> >
> ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml
> >
> >
> ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml
> > [root@na-nyc-bos-cs-poc2 /]# md5sum
> >
> ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml
> > baf5f63f5dd57484e9a88be6689939fc
> >
>  ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml
> > [root@na-nyc-bos-cs-poc2 /]# md5sum
> >
> ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml
> > 913b4561258980f7b1b4c7cb3de261ba
> >
>  ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml
> > [root@na-nyc-bos-cs-poc2 /]#
> >
> >
> >
> > *HELP!!!*
> >
> > I need to get my cloud back up and running.
> >
> >
> >
> >
> > Best Regards,
> >
> >
> >
> > Adam Scarcella
> >
> >
> > On Tue, Jun 10, 2014 at 8:41 PM, Rafael Weingartner <
> > rafaelweingartner@gmail.com> wrote:
> >
> >> Nice, no doubts that others will run into this.
> >> But when they run into that problem, it is going to be just a query at
> >> Google to uncover this thread ;)
> >>
> >>
> >> On Tue, Jun 10, 2014 at 8:39 PM, Steve Searles <ssearles@zimcom.net>
> >> wrote:
> >>
> >> >  Rafael,
> >> >
> >> >  Ok, I seem to have found the smoking gun in the localhost.log
> >> >
> >> >  Caused by: java.sql.SQLException: Binary logging not possible.
> Message:
> >> > Transaction level 'READ-COMMITTED' in InnoDB is not safe for binlog
> mode
> >> > 'STATEMENT'
> >> > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073)
> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597)
> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529)
> >> > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990)
> >> > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
> >> > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2625)
> >> > at
> >> >
> >>
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119)
> >> > at
> >> >
> >>
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2415)
> >> > at
> >> >
> >>
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2333)
> >> > at
> >> >
> >>
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2318)
> >> > at
> >> >
> >>
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
> >> > at
> >> >
> >>
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
> >> > at com.cloud.utils.db.GenericDaoBase.update(GenericDaoBase.java:823)
> >> > ... 198 more
> >> > Jun 10, 2014 7:02:25 PM org.apache.catalina.core.ApplicationContext
> log
> >> > INFO: Shutting down log4j
> >> >
> >> >  Oddly enough I thought the default logging type was MIXED not
> >> STATEMENT.
> >> >  I am sure others will run into this. I am running (5.1.73) This is
> what
> >> > the MySQL website had to say.
> >> >
> >> >           The default binary logging format depends on the version of
> >> > MySQL you are using:
> >> >
> >> >    -
> >> >
> >> >    For MySQL 5.1.11 and earlier, and for MySQL 5.1.29 and later,
> >> >    statement-based logging is used by default.
> >> >    -
> >> >
> >> >    For MySQL 5.1.12 through MySQL 5.1.28, mixed logging is used by
> >> default
> >> >
> >> >             If you are using InnoDB
> >> >    <http://dev.mysql.com/doc/refman/5.1/en/innodb-storage-engine.html
> >
> >>
> >> >    tables and the transaction isolation level is READ COMMITTED
> >> >    <
> >>
> http://dev.mysql.com/doc/refman/5.1/en/set-transaction.html#isolevel_read-committed
> >> >
> >> >    or READ UNCOMMITTED
> >> >    <
> >>
> http://dev.mysql.com/doc/refman/5.1/en/set-transaction.html#isolevel_read-uncommitted
> >> >,
> >> >    only row-based logging can be used. It is *possible* to change the
> >>
> >> >    logging format to STATEMENT, but doing so at runtime leads very
> >> >    rapidly to errors because InnoDB can no longer perform inserts.
> >> >
> >> >    Switching the replication format at runtime is not recommended when
> >> >    any temporary tables exist, because temporary tables are logged
> only
> >> when
> >> >    using statement-based replication, whereas with row-based
> >> replication they
> >> >    are not logged. With mixed replication, temporary tables are
> usually
> >> >    logged; exceptions happen with user-defined functions (UDFs) and
> >> with the
> >> >    UUID()
> >> >    <
> >>
> http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html#function_uuid
> >> >
> >>
> >> >    function.
> >> >
> >> >    With the binary log format set to ROW, many changes are written to
> >> the
> >> >    binary log using the row-based format. Some changes, however, still
> >> use the
> >> >    statement-based format. Examples include all DDL (data definition
> >> language)
> >> >    statements such as CREATE TABLE
> >> >    <http://dev.mysql.com/doc/refman/5.1/en/create-table.html>, ALTER
> >> TABLE
> >> >    <http://dev.mysql.com/doc/refman/5.1/en/alter-table.html>, or DROP
> >> >    TABLE <http://dev.mysql.com/doc/refman/5.1/en/drop-table.html>.
> >>
> >> >
> >> >
> >> >
> >> >  Add
> >> >  binlog-format=ROW
> >> > to /etc/my.cnf on the master DB server and you are good to go.
> >> >
> >> >
> >> >
> >> >    Steven Searles, CTO | ssearles@zimcom.net
> >> >  Zimcom Internet Solutions  | www.zimcom.net
> >> >  O: 513.231.9500  |  D: 513.233.4130
> >> >
> >> >
> >> >  On Jun 10, 2014, at 6:18 PM, Rafael Weingartner <
> >> > rafaelweingartner@gmail.com> wrote:
> >> >
> >> > maybe, but, I actually do not care that much about the app that is
> used
> >> to
> >> > integrate CS with AWS, hence they are two separated apps, a problem
> with
> >> > AWS integration app should not affect the CS normal app.
> >> >
> >> > Could you please provide me again with the log file from
> >> > /var/log/cloudstack/management/management-server.log? There must be
> >> > something here.
> >> >
> >> >
> >> > On Tue, Jun 10, 2014 at 6:34 PM, Steve Searles <ssearles@zimcom.net>
> >> > wrote:
> >> >
> >> > Is
> >> >
> >> >
> >>
> http://mail-archives.apache.org/mod_mbox/cloudstack-issues/201401.mbox/%3CJIRA.12687852.1389186771873.61435.1389186832715@arcas%3E
> >> >  relevant here?
> >> >
> >> >
> >> >
> >> > On Jun 10, 2014, at 5:31 PM, Steve Searles <ssearles@zimcom.net
> <mailto:
> >> > ssearles@zimcom.net>> wrote:
> >> >
> >> > Ok I was wrong, the addressing-1.5.4.mar is present.
> >> >
> >> > [root@csmgmt01 modules]# pwd
> >> > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules
> >> > [root@csmgmt01 modules]# ls -lash
> >> > total 344K
> >> > 4.0K drwxr-xr-x. 2 cloud cloud 4.0K Mar 31 18:21 .
> >> > 4.0K drwxr-xr-x. 7 cloud cloud 4.0K Mar 31 18:21 ..
> >> > 124K -rw-r--r--. 1 cloud cloud 124K Mar 27 01:06 addressing-1.5.4.mar
> >> > 36K -rw-r--r--. 1 cloud cloud  34K Mar 27 01:06 addressing-1.5.6.mar
> >> > 16K -rw-r--r--. 1 cloud cloud  15K Mar 27 01:06
> >> axis2-jaxws-mar-1.5.6.mar
> >> > 40K -rw-r--r--. 1 cloud cloud  38K Mar 27 01:06 mex-1.5.6.mar
> >> > 4.0K -rw-r--r--. 1 cloud cloud  139 Mar 27 01:06 modules.list
> >> > 20K -rw-r--r--. 1 cloud cloud  19K Mar 27 01:06 mtompolicy-1.5.6.mar
> >> > 20K -rw-r--r--. 1 cloud cloud  18K Mar 27 01:06 ping-1.5.6.mar
> >> > 12K -rw-r--r--. 1 cloud cloud 9.5K Mar 27 01:06 rahas-1.5.1.mar
> >> > 12K -rw-r--r--. 1 cloud cloud 9.4K Mar 27 01:06 rampart-1.5.1.mar
> >> > 36K -rw-r--r--. 1 cloud cloud  33K Mar 27 01:06 scripting-1.5.6.mar
> >> > 16K -rw-r--r--. 1 cloud cloud  16K Mar 27 01:06 soapmonitor-1.5.6.mar
> >> > [root@csmgmt01 modules]#
> >> >
> >> >
> >> >
> >> > The error is complaining about the module.xml file missing.
> >> >
> >> > Steve Searles
> >> >
> >> > On Jun 10, 2014, at 5:21 PM, Steve Searles <ssearles@zimcom.net
> <mailto:
> >> > ssearles@zimcom.net><mailto:ssearles@zimcom.net>> wrote:
> >> >
> >> > I agree with what you are saying.  Yes. addressing-1.5.4.mar errors I
> >> have
> >> > seen before.  I have no webapps7080 directory so that is why that
> error
> >> is
> >> > being thrown.  I still am wondering even if a dependency is missing
> why
> >> is
> >> > it only a dependency when sql replication(bin log) is on? Can you tell
> >> me
> >> > how to get a stack trace from CS startup?
> >> >
> >> > That file is present in these locations on the management node.
> >> >
> >> > [root@csmgmt01 /]# find / -name addressing-1.5.4.mar
> >> >
> >>
> /root/.m2/repository/org/apache/axis2/addressing/1.5.4/addressing-1.5.4.mar
> >> >
> >> >
> >> >
> >>
> /root/apache-cloudstack-4.3.0-src/dist/rpmbuild/BUILD/cloudstack-4.3.0/awsapi/target/cloud-awsapi-4.3.0/WEB-INF/modules/addressing-1.5.4.mar
> >> >
> >> >
> >> >
> >>
> /usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/modules/addressing-1.5.4.mar
> >> > [root@csmgmt01 /]#
> >> >
> >> >
> >> >
> >> > is it just a symlink that is missing?
> >> >
> >> >
> >> >
> >> > [root@csmgmt01 webapps]# ls -las
> >> > total 12
> >> > 4 drwxr-xr-x.  3 root root 4096 Mar 31 18:21 .
> >> > 4 drwxr-xr-x.  4 root root 4096 Mar 31 18:21 ..
> >> > 4 drwxr-xr-x. 11 root root 4096 Mar 31 18:21 client
> >> > [root@csmgmt01 webapps]# cd client/
> >> > [root@csmgmt01 client]# ls
> >> > css  dictionary.jsp  images  index.jsp  lib  META-INF  modules
>  plugins
> >> > scripts  tests  WEB-INF
> >> > [root@csmgmt01 client]# ls -lash
> >> > total 264K
> >> > 4.0K drwxr-xr-x. 11 root root 4.0K Mar 31 18:21 .
> >> > 4.0K drwxr-xr-x.  3 root root 4.0K Mar 31 18:21 ..
> >> > 4.0K drwxr-xr-x.  2 root root 4.0K Mar 31 18:21 css
> >> > 116K -rw-r--r--.  1 root root 115K Mar 27 01:06 dictionary.jsp
> >> > 4.0K drwxr-xr-x.  2 root root 4.0K Mar 31 18:21 images
> >> > 104K -rw-r--r--.  1 root root 103K Mar 27 01:06 index.jsp
> >> > 4.0K drwxr-xr-x.  5 root root 4.0K Mar 31 18:21 lib
> >> > 4.0K drwxr-xr-x.  2 root root 4.0K Mar 27 01:06 META-INF
> >> > 4.0K drwxr-xr-x.  6 root root 4.0K Mar 31 18:21 modules
> >> > 4.0K drwxr-xr-x.  3 root root 4.0K Mar 31 18:21 plugins
> >> > 4.0K drwxr-xr-x.  4 root root 4.0K Mar 31 18:21 scripts
> >> > 4.0K drwxr-xr-x.  2 root root 4.0K Mar 31 18:21 tests
> >> > 4.0K drwxr-xr-x.  4 root root 4.0K Mar 31 18:21 WEB-INF
> >> > [root@csmgmt01 client]# pwd
> >> > /usr/share/cloudstack-management/webapps/client
> >> > [root@csmgmt01 client]#
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > Steve Searles
> >> >
> >> >
> >> > On Jun 10, 2014, at 3:04 PM, Rafael Weingartner <
> >> > rafaelweingartner@gmail.com<mailto:rafaelweingartner@gmail.com
> ><mailto:
> >> > rafaelweingartner@gmail.com><mailto:rafaelweingartner@gmail.com>>
> >> wrote:
> >> >
> >> > Look at this point:
> >> > "SEVERE: Error listenerStart
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext start
> >> > SEVERE: Context [/client] startup failed due to previous errors"
> >> > I could not find the stacktrace into the logs you provided. But, it
> has
> >> to
> >> > have some stack which would help to understand why the CS is not
> getting
> >> > up.
> >> >
> >> > This is a good point to look at:
> >> > "SEVERE: The addressing-1.5.4.mar module, which is not valid, caused
> The
> >> > module.xml file cannot be found for the module:
> >> >
> >> >
> >> >
> >>
> /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar
> >> > org.apache.axis2.deployment.DeploymentException: The module.xml file
> >> cannot
> >> > be found for the module:
> >> >
> >> >
> >> >
> >>
> /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar
> >> > at
> >> >
> >> >
> >> >
> >>
> org.apache.axis2.deployment.repository.util.ArchiveReader.readModuleArchive(ArchiveReader.java:529)
> >> > at
> >> >
> >>
> org.apache.axis2.deployment.ModuleDeployer.deploy(ModuleDeployer.java:75)
> >> > at
> >> >
> >> >
> >> >
> >>
> org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)
> >> > at
> >> >
> >> >
> >> >
> >>
> org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:661)"
> >> >
> >> >
> >> >
> >> > It seems there is a .xml file being missing.
> >> > But, still even though the awsapi is not working, the CS app should
> run
> >> > just fine, unless it depends on that service.
> >> >
> >> > This is also something interesting:
> >> > "Jun 10, 2014 2:12:58 PM org.apache.catalina.loader.WebappClassLoader
> >> > loadClass
> >> > INFO: Illegal access: this web application instance has been stopped
> >> > already.  Could not load
> >> > org.apache.cloudstack.managed.context.ManagedContextTimerTask$1.  The
> >> > eventual following stack trace is caused by an error thrown for
> >> debugging
> >> > purposes as well as to attempt to terminate the thread which caused
> the
> >> > illegal access, and has no functional impact.
> >> > java.lang.IllegalStateException
> >> > at
> >> >
> >> >
> >> >
> >>
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1369)"
> >> >
> >> > it seems there is a missing dependency.
> >> >
> >> > BTW: out of curiosity, what do tou have in folder
> >> > "/usr/share/cloudstack-management/webapps"
> >> >
> >> >
> >> > On Tue, Jun 10, 2014 at 3:53 PM, Steve Searles <ssearles@zimcom.net
> >> > <mailto:ssearles@zimcom.net><mailto:ssearles@zimcom.net><mailto:
> >> > ssearles@zimcom.net>> wrote:
> >> >
> >> > Nothing interesting in the management server logs, you saw the
> complete
> >> > ones I sent yesterday.  I have attached the complete catalina.out
> below.
> >> > This is starting to feel like a problem with the DB locking the tables
> >> > during the startup integrity check.  I am able to lock the tables via
> >> the
> >> > command line with the bin log enabled without issue though.
> >> >
> >> > https://dl.dropboxusercontent.com/u/51895629/catalina.binlog.gz
> >> >
> >> > Steve Searles
> >> >
> >> >
> >> > On Jun 10, 2014, at 2:29 PM, Rafael Weingartner <
> >> > rafaelweingartner@gmail.com<mailto:rafaelweingartner@gmail.com>>
> wrote:
> >> >
> >> > this is interesting:
> >> > "
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext start
> >> > SEVERE: Context [/client] startup failed due to previous error"
> >> >
> >> > does it have any extra log before that message?
> >> > Did you find anything at /var/log/cloudstack/management-server.log?
> >> >
> >> >
> >> > On Tue, Jun 10, 2014 at 3:24 PM, Steve Searles <ssearles@zimcom.net
> >> > <mailto:ssearles@zimcom.net>>
> >> > wrote:
> >> >
> >> > I found the following differences in the catalina.out file with bin
> log
> >> > enabled and disabled.  Can you point me in the right direction on
> >> debugging
> >> > the actual /client application further.  Obviously the problem is
> >> >
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext start
> >> > SEVERE: Error listenerStart
> >> >
> >> > I could use a point in the right direction to get further debugging
> >> > info.
> >> >
> >> >
> >> > WITH BINLOG
> >> >
> >> > INFO  [c.c.u.c.ComponentContext] (main:null) Setup Spring Application
> >> > context
> >> > INFO  [c.c.s.GsonHelper] (main:null) Default Builder inited.
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext start
> >> > SEVERE: Error listenerStart
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext start
> >> > SEVERE: Context [/client] startup failed due to previous errors
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader
> >> > clearReferencesThreads
> >> > SEVERE: A web application appears to have started a thread named
> >> > [FileWatchdog] but has failed to stop it. This is very likely to
> create
> >> a
> >> > memory leak.
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader
> >> > clearReferencesThreads
> >> > SEVERE: A web application appears to have started a thread named
> >> [Timer-0]
> >> > but has failed to stop it. This is very likely to create a memory
> leak.
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader
> >> > clearReferencesThreads
> >> > SEVERE: A web application appears to have started a thread named
> >> > [ConnectionConcierge-1] but has failed to stop it. This is very likely
> >> to
> >> > create a memory leak.
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader
> >> > clearReferencesThreads
> >> > SEVERE: A web application appears to have started a thread named
> >> > [ClusteredAgentManager Timer] but has failed to stop it. This is very
> >> > likely to create a memory leak.
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader
> >> > clearReferencesThreads
> >> > SEVERE: A web application appears to have started a thread named
> >> [Timer-1]
> >> > but has failed to stop it. This is very likely to create a memory
> leak.
> >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader
> >> > clearThreadLocalMap
> >> > SEVERE: A web application created a ThreadLocal with key of type
> >> > [org.springframework.core.NamedThreadLocal] (value [Prototype beans
> >> > currently in creation]) and a value of type [null] (value [null]) but
> >> > failed to remove it when the web application was stopped. To prevent a
> >> > memory leak, the ThreadLocal has be
> >> > en forcibly removed.
> >> >
> >> >
> >> > WITHOUT BINLOG
> >> >
> >> >
> >> > INFO  [c.c.u.c.ComponentContext] (main:null) Setup Spring Application
> >> > context
> >> > INFO  [c.c.s.GsonHelper] (main:null) Default Builder inited.
> >> > INFO  [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) Running
> system
> >> > integrity checker com.cloud.upgrade.DatabaseIntegrityChecker@7f2af4a3
> >> > INFO  [c.c.u.DatabaseIntegrityChecker] (main:null) Grabbing lock to
> >> check
> >> > for database integrity.
> >> > INFO  [c.c.u.DatabaseIntegrityChecker] (main:null) Performing database
> >> > integrity check
> >> > INFO  [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) Running
> system
> >> > integrity checker
> >> > org.apache.cloudstack.utils.identity.ManagementServerNode@7c73b9ad
> >> > INFO  [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) Configuring
> >> > CloudStack Components
> >> > INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name =
> >> > UserDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk =
> false
> >> > maxEntriesLocalHeap = 5000 maxEntriesLocalDisk = 0
> >> > memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 300
> >> timeToIdleSeconds =
> >> > 300 persistence = none diskExpiryThreadInt
> >> > ervalSeconds = 120 cacheEventListeners:
> >> > net.sf.ehcache.statistics.LiveCacheStatisticsWrapper  hitCount = 0
> >> > memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0
> >> > missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
> >> > maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
> >> > INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name =
> >> > VMTemplateDaoImpl status = STATUS_ALIVE eternal = false
> overflowToDisk =
> >> > false maxEntriesLocalHeap = 100 maxEntriesLocalDisk = 0
> >> > memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600
> >> timeToIdleSeconds =
> >> > 300 persistence = none diskExpiryThre
> >> > adIntervalSeconds = 120 cacheEventListeners:
> >> > net.sf.ehcache.statistics.LiveCacheStatisticsWrapper  hitCount = 0
> >> > memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0
> >> > missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
> >> > maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = fals
> >> > e ]
> >> > INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name =
> >> > ServiceOfferingDaoImpl status = STATUS_ALIVE eternal = false
> >> overflowToDisk
> >> > = false maxEntriesLocalHeap = 50 maxEntriesLocalDisk = 0
> >> > memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600
> >> timeToIdleSeconds =
> >> > 300 persistence = none diskExpiry
> >> > ThreadIntervalSeconds = 120 cacheEventListeners:
> >> > net.sf.ehcache.statistics.LiveCacheStatisticsWrapper  hitCount = 0
> >> > memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0
> >> > missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
> >> > maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned =
> >> > false ]
> >> > INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name =
> >> > DiskOfferingDaoImpl status = STATUS_ALIVE eternal = false
> >> overflowToDisk =
> >> > false maxEntriesLocalHeap = 50 maxEntriesLocalDisk = 0
> >> > memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600
> >> timeToIdleSeconds =
> >> > 300 persistence = none diskExpiryThr
> >> > eadIntervalSeconds = 120 cacheEventListeners:
> >> > net.sf.ehcache.statistics.LiveCacheStatisticsWrapper  hitCount = 0
> >> > memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0
> >> > missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
> >> > maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = fal
> >> > se ]
> >> >
> >> >
> >> >
> >> >
> >> > On Jun 10, 2014, at 10:30 AM, Rafael Weingartner <
> >> > rafaelweingartner@gmail.com<mailto:rafaelweingartner@gmail.com>>
> wrote:
> >> >
> >> > you do not need to debug into the mysql, you just need to find the
> stack
> >> > trace in the tomcat that is causing the app to shutdown during the
> >> start up
> >> > process
> >> >
> >> >
> >> > On Tue, Jun 10, 2014 at 11:26 AM, Steve Searles <ssearles@zimcom.net
> >> > <mailto:ssearles@zimcom.net>>
> >> > wrote:
> >> >
> >> > Yes, I will see if I can get any more debugging info out of it.  Yes
> >> that
> >> > is the URL we are using to access.  In the condition where the bin log
> >> is
> >> > on I get a 404 from tomcat meaning that the application has not
> >> started.  I
> >> > will try some further debugging on the mysql side as well.  This is
> not
> >> an
> >> > upgrade but rather a fresh install of 4.3.  That is the problem I am
> >> seeing
> >> > as well, no useful debug information.
> >> >
> >> > Steve Searles
> >> >
> >> > On Jun 10, 2014, at 8:21 AM, Rafael Weingartner <
> >> > rafaelweingartner@gmail.com<mailto:rafaelweingartner@gmail.com>>
> wrote:
> >> >
> >> > I thought that it could be a missing dependency problem, because with
> CS
> >> > 4.3 upgrade from 4.2.1 we had one. The mysql jdbc .jar was not in the
> >> > classpath. So, we had to manually add it.
> >> >
> >> > I did not see any expressive tty in your log files.
> >> > Could you look for some more meaningful java stack traces, then we
> might
> >> >
> >> > be
> >> >
> >> > able to help you.
> >> > BTW, how are you trying to access the CS? http://serverIp:8080/client
> ?
> >> >
> >> >
> >> > On Mon, Jun 9, 2014 at 11:31 PM, Steve Searles <ssearles@zimcom.net
> >> > <mailto:ssearles@zimcom.net>>
> >> >
> >> > wrote:
> >> >
> >> >
> >> > It is also worth noting that these are the last two DB queries that
> >> >
> >> > happen
> >> >
> >> > before the stop. So it may have something to do with cloudbridge?
> >> >
> >> >
> >> >
> >> > 140609 22:28:00     8 Connect   cloud@db01.cvg.cs.zimcom.net<mailto:
> >> > cloud@db01.cvg.cs.zimcom.net> on
> >> > cloudbridge
> >> >              8 Query     /* mysql-connector-java-5.1.17-SNAPSHOT (
> >> > Revision: ${bzr.revision-id} ) */SHOW VARIABLES WHERE Variable_name
> >> > ='language' OR Variable_name = 'net_write_timeout' OR Variable_name =
> >> > 'interactive_timeout' OR Variable_name = 'wait_timeout' OR
> >> >
> >> > Variable_name =
> >> >
> >> > 'character_set_client' OR Variable_name = 'character_set_connection'
> OR
> >> > Variable_name = 'character_set' OR Variable_name =
> >> >
> >> > 'character_set_server'
> >> >
> >> > OR Variable_name = 'tx_isolation' OR Variable_name =
> >> > 'transaction_isolation' OR Variable_name = 'character_set_results' OR
> >> > Variable_name = 'timezone' OR Variable_name = 'time_zone' OR
> >> >
> >> > Variable_name
> >> >
> >> > = 'system_time_zone' OR Variable_name = 'lower_case_table_names' OR
> >> > Variable_name = 'max_allowed_packet' OR Variable_name =
> >> >
> >> > 'net_buffer_length'
> >> >
> >> > OR Variable_name = 'sql_mode' OR Variable_name = 'query_cache_type' OR
> >> > Variable_name = 'query_cache_size' OR Variable_name = 'init_connect'
> >> >              8 Query     /* mysql-connector-java-5.1.17-SNAPSHOT (
> >> > Revision: ${bzr.revision-id} ) */SELECT
> >> >
> >> > @@session.auto_increment_increment
> >> >
> >> >              8 Query     SHOW COLLATION
> >> >              8 Query     SET NAMES latin1
> >> >              8 Query     SET character_set_results = NULL
> >> >              8 Query     SET autocommit=1
> >> >              8 Query     SET sql_mode='STRICT_TRANS_TABLES'
> >> >              8 Query     SET autocommit=0
> >> >              8 Query     SET autocommit=0
> >> >              8 Query     SELECT offering_bundle.ID,
> >> > offering_bundle.AmazonEC2Offering, offering_bundle.CloudStackOffering
> >> >
> >> > FROM
> >> >
> >> > offering_bundle
> >> >              8 Query     commit
> >> >              8 Query     rollback
> >> >              8 Query     SET autocommit=1
> >> >              8 Query     SET autocommit=0
> >> >              8 Query     SET autocommit=0
> >> >              8 Query     SELECT mhost.ID, mhost.MHostKey,
> >> > mhost.Host, mhost.Version, mhost.LastHeartbeatTime FROM mhost WHERE
> >> > mhost.MHostKey = _binary'00:50:56:92:04:18'  ORDER BY RAND() LIMIT 1
> >> >              8 Query     commit
> >> >              8 Query     rollback
> >> >              8 Query     SET autocommit=1
> >> >              8 Query     SET autocommit=0
> >> >              8 Query     SET autocommit=0
> >> >              8 Query     UPDATE mhost SET mhost.Host=_binary'
> >> > http://localhost:7080/awsapi' WHERE mhost.ID = 1
> >> >              8 Query     commit
> >> >              8 Query     rollback
> >> >              8 Query     SET autocommit=1
> >> >              8 Query     SET autocommit=0
> >> >              8 Query     SET autocommit=0
> >> >              8 Query     SELECT shost.ID, shost.Host,
> >> > shost.HostType, shost.MHostID, shost.ExportRoot, shost.UserOnHost,
> >> > shost.UserPassword FROM shost WHERE shost.MHostID = 1  AND
> >> >
> >> > shost.ExportRoot
> >> >
> >> > = _binary'/Users/john1/S3-Mount'  ORDER BY RAND() LIMIT 1
> >> >              8 Query     commit
> >> >              8 Query     rollback
> >> >              8 Query     SET autocommit=1
> >> >              9 Connect   cloud@db01.cvg.cs.zimcom.net<mailto:
> >> > cloud@db01.cvg.cs.zimcom.net> on cloud
> >> >              9 Query     /* mysql-connector-java-5.1.17-SNAPSHOT (
> >> > Revision: ${bzr.revision-id} ) */SHOW VARIABLES WHERE Variable_name
> >> > ='language' OR Variable_name = 'net_write_timeout' OR Variable_name =
> >> > 'interactive_timeout' OR Variable_name = 'wait_timeout' OR
> >> >
> >> > Variable_name =
> >> >
> >> > 'character_set_client' OR Variable_name = 'character_set_connection'
> OR
> >> > Variable_name = 'character_set' OR Variable_name =
> >> >
> >> > 'character_set_server'
> >> >
> >> > OR Variable_name = 'tx_isolation' OR Variable_name =
> >> > 'transaction_isolation' OR Variable_name = 'character_set_results' OR
> >> > Variable_name = 'timezone' OR Variable_name = 'time_zone' OR
> >> >
> >> > Variable_name
> >> >
> >> > = 'system_time_zone' OR Variable_name = 'lower_case_table_names' OR
> >> > Variable_name = 'max_allowed_packet' OR Variable_name =
> >> >
> >> > 'net_buffer_length'
> >> >
> >> > OR Variable_name = 'sql_mode' OR Variable_name = 'query_cache_type' OR
> >> > Variable_name = 'query_cache_size' OR Variable_name = 'init_connect'
> >> >              9 Query     /* mysql-connector-java-5.1.17-SNAPSHOT (
> >> > Revision: ${bzr.revision-id} ) */SELECT
> >> >
> >> > @@session.auto_increment_increment
> >> >
> >> >              9 Query     SHOW COLLATION
> >> >              9 Query     SET NAMES latin1
> >> >              9 Query     SET character_set_results = NULL
> >> >              9 Query     SET autocommit=1
> >> >              9 Query     SET sql_mode='STRICT_TRANS_TABLES'
> >> >              9 Query     SET autocommit=0
> >> >              9 Query     SELECT @@session.tx_isolation
> >> >              9 Query     SET SESSION TRANSACTION ISOLATION LEVEL
> >> > READ COMMITTED
> >> >              9 Query     SELECT 1
> >> >              9 Query     SET autocommit=0
> >> >              9 Query     SELECT configuration.name,
> >> > configuration.value FROM configuration WHERE configuration.name =
> >> > _binary'enable.ec2.api'  ORDER BY RAND() LIMIT 1
> >> >              9 Query     commit
> >> >              9 Query     rollback
> >> >              9 Query     SET autocommit=1
> >> >
> >> >
> >> >
> >> > Steve Searles
> >> >
> >> >
> >> >
> >> > -----Original Message-----
> >> > From: Steve Searles [mailto:ssearles@zimcom.net <ssearles@zimcom.net
> >> > <mailto:ssearles@zimcom.net>> <
> >> > ssearles@zimcom.net<mailto:ssearles@zimcom.net>>]
> >> > Sent: Monday, June 9, 2014 10:19 PM
> >> > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>
> >> > Subject: RE: Management Server Startup Issue
> >> >
> >> > Rafael, Thank you for responding so quickly.  Yes 4.3.0. I am using a
> >> > dedicated DB servers and dedicated management servers.  I will provide
> >> > whatever log files you would find useful. It would be odd if it was a
> >> > dependancy since CS really has no visibility to the sql server other
> >> >
> >> > than a
> >> >
> >> > connect string? Or so I thought; and the system is running fine with
> the
> >> > binlog off.  With the bin log on I can run queries from the command
> line
> >> > and everything seems normal from the sql end,  but the management
> server
> >> > will not run.  Toggle binlog off CS works fine.  The replication slave
> >> >
> >> > has
> >> >
> >> > not even been started yet.  Let me know what other info I can send
> over
> >> > that would be of help.  There is nothing in the catalina logs of
> >> >
> >> > interest
> >> >
> >> > and my logging level is currently set to trace for the
> >> > management-server.log.  I will turn on the sql query log and compare
> >> >
> >> > them
> >> >
> >> > between startups of the management nodes perhaps that will yeild
> >> >
> >> > something.
> >> >
> >> > I have included a dropbox link with the management server log from a
> >> > normal startup of CS and one with mysql binlog enabled.  Any help
> would
> >> >
> >> > be
> >> >
> >> > greatly appriciated.
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> https://dl.dropboxusercontent.com/u/51895629/management-server.logs.tar.gz
> >> >
> >> >
> >> >
> >> >
> >> > Steve Searles
> >> >
> >> >
> >> > -----Original Message-----
> >> > From: Rafael Weingartner [mailto:rafaelweingartner@gmail.com]
> >> > Sent: Monday, June 9, 2014 6:57 PM
> >> > To: users@cloudstack.apache.org
> >> > Subject: Re: Management Server Startup Issue
> >> >
> >> > please give more logs.
> >> > It is the 4.3.0 right? I bet there is a problem with a missing
> >> >
> >> > dependency.
> >> >
> >> >
> >> >
> >> > On Mon, Jun 9, 2014 at 7:32 PM, Steve Searles <ssearles@zimcom.net>
> >> >
> >> > wrote:
> >> >
> >> >
> >> > Anyone have any idea why the management service would refuse to start
> >> > if the DB server is configured as a master for replication?
> >> >
> >> >
> >> >
> >> > log_bin=mysql-bin
> >> > server_id=1
> >> >
> >> > Adding the above the my.cnf file of my DB server, restarting mysql
> >> > then restarting cloudstack-managment will cause cloudstack to get hung
> >> > on startup here.
> >> >
> >> > 2014-06-09 18:25:33,769 INFO  [c.c.u.c.ComponentContext] (main:null)
> >> > Starting
> >> > com.cloud.bridge.persist.dao.SHostDaoImpl_EnhancerByCloudStack_9da5499
> >> > b
> >> > 2014-06-09 18:25:33,769 INFO  [c.c.u.c.ComponentContext] (main:null)
> >> > Starting
> >> > com.cloud.bridge.persist.dao.SObjectDaoImpl_EnhancerByCloudStack_847fb
> >> > 98c
> >> > 2014-06-09 18:25:33,769 INFO  [c.c.u.c.ComponentContext] (main:null)
> >> > Starting
> >> > com.cloud.bridge.persist.dao.CloudStackUserDaoImpl_EnhancerByCloudStac
> >> > k_7fce3ee6
> >> > 2014-06-09 18:25:33,769 INFO  [c.c.u.c.ComponentContext] (main:null)
> >> > Starting
> >> > com.cloud.bridge.service.core.ec2.EC2Engine_EnhancerByCloudStack_d70c9
> >> > e3c
> >> > 2014-06-09 18:25:33,769 INFO  [c.c.u.c.ComponentContext] (main:null)
> >> > Starting
> >> > com.cloud.bridge.service.controller.s3.ServiceProvider_EnhancerByCloud
> >> > Stack_23d38b1
> >> >
> >> > That is the last few lines of the management-server.log and it will
> >> > hang there indefinitely.
> >> >
> >> > I noticed this on startup as well.
> >> >
> >> > 2014-06-09 18:25:16,841 INFO  [c.c.u.d.T.Transaction] (main:null) Is
> >> > Data Base High Availiability enabled? Ans : false
> >> > 2014-06-09 18:25:17,164 DEBUG [c.c.u.d.ConnectionConcierge]
> >> > (main:null) Registering a database connection for LockMaster1
> >> > 2014-06-09 18:25:17,164 INFO  [c.c.u.d.Merovingian2] (main:null)
> >> > Cleaning up locks for 345049793560
> >> > 2014-06-09 18:25:17,175 INFO  [c.c.u.d.Merovingian2] (main:null)
> >> > Released
> >> > 0 locks for 345049793560
> >> > 2014-06-09 18:25:17,206 INFO  [o.a.c.s.l.CloudStackExtendedLifeCycle]
> >> > (main:null) Running system integrity checker
> >> > com.cloud.upgrade.DatabaseUpgradeChecker@63c8e4ae
> >> > 2014-06-09 18:25:17,207 INFO  [c.c.u.DatabaseUpgradeChecker]
> >> > (main:null) Grabbing lock to check for database upgrade.
> >> > 2014-06-09 18:25:17,254 DEBUG [c.c.u.d.VersionDaoImpl] (main:null)
> >> > Checking to see if the database is at a version before it was the
> >> > version table is created
> >> > 2014-06-09 18:25:17,295 INFO  [c.c.u.DatabaseUpgradeChecker]
> >> > (main:null) DB version = 4.3.0 Code Version = 4.3.0
> >> > 2014-06-09 18:25:17,296 INFO  [c.c.u.DatabaseUpgradeChecker]
> >> > (main:null) DB version and code version matches so no upgrade needed.
> >> >
> >> > The management server talking to the DB server at least enough to
> >> > check the version; and mysql from the command line on the management
> >> > server works fine as well.
> >> >
> >> >
> >> > This line caught my attention.
> >> >
> >> >
> >> > 2014-06-09 18:25:16,841 INFO  [c.c.u.d.T.Transaction] (main:null) Is
> >> > Data Base High Availiability enabled? Ans : false
> >> >
> >> > Is there something I need to set when using mysql replication specific
> >> > to cloudstack? I do not see anything in the documentation on this
> >> > other than setting up basic replication.
> >> >
> >> >
> >> > if I comment out the above two lines in my.cnf and restart the
> >> > management server it fires up just fine.  I don’t get how enabling bin
> >> > log would make a difference at all.
> >> >
> >> > Any help or explication for this strange issue would be appreciated.
> >> >
> >> >
> >> > - Steve Searles
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Rafael Weingärtner
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Rafael Weingärtner
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Rafael Weingärtner
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Rafael Weingärtner
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Rafael Weingärtner
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Rafael Weingärtner
> >> >
> >> >
> >> >
> >>
> >>
> >> --
> >> Rafael Weingärtner
> >>
> >
> >
>
>
> --
> Rafael Weingärtner
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message