tomee-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zachary Bedell <zbed...@nycourts.gov>
Subject Re: Get JDBC pool capacity
Date Wed, 02 Nov 2016 20:34:05 GMT
I've narrowed this down a little more.  It appears that using an XA DataSource versus a non-XA
Driver class is what switches the behavior.  If I just switch to using the non-XA Oracle driver,
I can take out DataSourceCreater and still get stats in JMX.

So both of these give me stats:

JdbcDriver oracle.jdbc.xa.client.OracleXADataSource
DataSourceCreator dbcp

-OR-

JdbcDriver oracle.jdbc.OracleDriver
# DataSourceCreator not set
# Though of course no XA for this one...


Digging a little deeper, it looks like the datasource settings I had weren't enabling pooling
with the default TomEEDataSourceCreator, but were with DbcpDataSourceCreator, though I'm not
sure why the difference there.


I had my pool sizes set using InitialSize & MaxActive in resources.xml.  Based on [http://tomee.apache.org/datasource-config.html],
those seemed to be the right attributes.  Looking in org.apache.openejb.resource.jdbc.DataSourceFactory#usePool(),
it would determine to NOT use a pool unless initialPoolSize and maxPoolSize were defined in
the datasource resource instead of InitialSize and MaxActive.

It looks like DataSourceFactory::create() has different logic depending on whether the class
specified in the JdbcDriver attribute is a DataSource or a Driver.  Looks like with a Driver,
create() always calls DataSourceCreator#pool{,Managed,ManagedWIthRecovery}(), but with a DataSource,
usePool has to return TRUE for that to happen.

So this works w/ stats in JMX AND pooling:

    JdbcDriver oracle.jdbc.xa.client.OracleXADataSource
    JdbcUrl ...
    User ...
    UserName ...
    VaultPassword ...
    InitialSize 0
    MaxActive 10
    InitialPoolSize 0 # Note: Adding this
    MaxPoolSize 10 # .. And make the difference
    AccessToUnderlyingConnectionAllowed true
    JtaManaged true

It's using the Oracle XA DataSource and the default TomEEDataSourceCreator class (not forced
to dbcp), and I still get stats.

Google doesn't show the MaxPoolSize attribute anywhere on tomee.apache.org<http://tomee.apache.org>,
so it seems like that shouldn't be necessary, but I'm not really sure what's right at this
point.

-Zac


On Oct 31, 2016, at 15:18, Romain Manni-Bucau <rmannibucau@gmail.com<mailto:rmannibucau@gmail.com>>
wrote:

2016-10-31 20:08 GMT+01:00 Zachary Bedell <zbedell@nycourts.gov<mailto:zbedell@nycourts.gov>>:

We're not a maven shop, alas...  Ant and lots of Ivy magic to get a build
working outside.



I *suspect* it's either the datasource-in-ear or the custom implementation
of org.apache.openejb.cipher.PasswordCipher we're using that's the
cause.  I'll see if I can steal some time this week to factor those two out
to confirm one way or the other & let you know.


If we are on the bet side of things I'll bet neither of both since the
ciphering happens in a completely part of the code and ear or not it should
reuse the same factory so jmx registration. Testing with a war or just
removing it from the ear and using tomee.xml can confirm it or not BTW.


-Zac

On Oct 31, 2016, at 14:33, Romain Manni-Bucau <rmannibucau@gmail.com<mailto:rmannibucau@gmail.com>>
wrote:

if you can share a project i can run with "mvn package tomee:run" I can
sort it out for you if it helps


Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://blog-rmannibucau.rhcloud.com> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/
rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory
<https://javaeefactory-rmannibucau.rhcloud.com>

2016-10-31 18:27 GMT+01:00 Zachary Bedell <zbedell@nycourts.gov<mailto:zbedell@nycourts.gov>>:

I tried some variations just to get an idea of what's actually the magic
thing that's making my config (not) work.

1)
resource.xml:
      DataSourceCreator dbcp
      # JmxEnabled absent -- default value

system.properties:
      No setting for datasource-creator

RESULT: Works.  Attributes visible in JMX under "openejb.management:
DataSource=UCMSSecurity/ucsSecurityKeystore,ObjectType=datasources"


2)
resource.xml:
      # DataSourceCreator absent -- default value
      JmxEnabled true

system.properties:
      No setting for datasource-creator

RESULT: Fails.  Nothing in JMX for the datasources.


3)
resource.xml:
      # DataSourceCreator absent -- default value
      # JmxEnabled -- default value

system.properties:
      openejb.jdbc.datasource-creator = *anything*

RESULT: Fails.

I tried dbcp, org.apache.tomee.jdbc.TomEEDataSourceCreator,
tomcat-dbcp,
as well as complete gibberish for the setting in system.properties.  In
all
cases, the log message came out as:

13:07:18,631 INFO  [options] Using 'openejb.jdbc.datasource-
creator=org.apache.tomee.jdbc.TomEEDataSourceCreator'

(I'm using log4j, thus different formatting on the logs.)

The string "datasource-creator" doesn't appear anywhere in my
distribution
other than system.properties and the log files, so I don't see how
anything
else could be setting that value.  The setting in system.properties
appears
to be ignored.


I can run with case #1 now, explicitly setting the datasource creator in
each datasource.  I'm getting the stats I need that way.  If there's any
other info I can provide to figure out why the setting is being ignored,
let me know.

Thanks again for the help!

-Zac



On Oct 28, 2016, at 17:08, Romain Manni-Bucau <rmannibucau@gmail.com<mailto:rmannibucau@gmail.com>>
wrote:

still openejb.jdbc.datasource-creator and it should even be logged
during
startup:

Feb 07, 2014 10:40:05 PM org.apache.openejb.util.OptionsLog info
INFO: Using 'openejb.jdbc.datasource-creator=org.apache.tomee.jdbc.
TomEEDataSourceCreator'

you can give a try to add jmxEnabled=true to your datasource properties
but
should be the default



Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://blog-rmannibucau.rhcloud.com> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/
rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory
<https://javaeefactory-rmannibucau.rhcloud.com>

2016-10-28 23:04 GMT+02:00 Zachary Bedell <zbedell@nycourts.gov<mailto:zbedell@nycourts.gov>>:

As far as I know, I didn't do anything to change the pooling
implementation.

It does all start working if I add an explicit "DataSourceCreator
dbcp"
to
my datasource definitions in resource.xml.  With that, the datasources
section shows up in JMX, and I see all the stats I need.

I found a suggestion on this page (https://rmannibucau.
wordpress.com/2012/08/10/switching-of-datasource-<http://wordpress.com/2012/08/10/switching-of-datasource->
connection-pooling-in-
tomeeopenejb/) that adding "openejb.jdbc.datasource-creator = dbcp"
to
system.properties should do the same globally, but that didn't work
for
me.  Adding that & removing the DataSourceCreator entry went back to
no
stats.  Maybe there's any updated name for that property?

Not sure what about my config breaks the default configuration, but
this
looks like it should do the trick for now.

Thanks for the pointers!

-Zac


On Oct 28, 2016, at 16:14, Romain Manni-Bucau <rmannibucau@gmail.com<mailto:rmannibucau@gmail.com>
<mailto:rmannibucau@gmail.com>> wrote:

weird, try to reproduce your setup on github I'll check next week

only case I'm thiking about is you don't use dbcp or tomcat-jdbc
pooling


Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://blog-rmannibucau.rhcloud.com<https://blog-
rmannibucau.rhcloud.com/<http://rmannibucau.rhcloud.com/>>> | Old Blog
<http://rmannibucau.wordpress.com<http://rmannibucau.wordpress.com/>>
|
Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory
<https://javaeefactory-rmannibucau.rhcloud.com<https:
//javaeefactory-rmannibucau.rhcloud.com/<http://javaeefactory-rmannibucau.rhcloud.com/>>>

2016-10-28 22:12 GMT+02:00 Zachary Bedell <zbedell@nycourts.gov<mailto:zbedell@nycourts.gov>
<mailto:
z
bedell@nycourts.gov<mailto:bedell@nycourts.gov>>>:

That doesn't exist for me at all in jconsole.

Screenshot: http://imgur.com/a/5mMgz

If I enumerate JMX using an ObjectName of "openejb.management:*" with
something like the code below, there's no "datasources" in any of the
found
objects.  Nothing matching the names of my pools either.

final ObjectName name = new ObjectName("openejb.management:*");
final Set<ObjectInstance> search = mbeanServer.queryMBeans(name,
null);
for(final ObjectInstance objectInstance : search) {
System.out.println(objectInstance.getObjectName()
.getCanonicalName());
}

-Zac

On Oct 28, 2016, at 15:55, Romain Manni-Bucau <rmannibucau@gmail.com<mailto:rmannibucau@gmail.com>
<mailto:rmannibucau@gmail.com>
<mailto:rmannibucau@gmail.com>> wrote:

openejb.management > datasources



Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://blog-rmannibucau.rhcloud.com<https://blog-
rmannibucau.rhcloud.com/<http://rmannibucau.rhcloud.com/>>> | Old Blog
<http://rmannibucau.wordpress.com<http://rmannibucau.wordpress.com/>>
|
Github <https://github.com/
rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory
<https://javaeefactory-rmannibucau.rhcloud.com<https:
//javaeefactory-rmannibucau.rhcloud.com/<http://javaeefactory-rmannibucau.rhcloud.com/>>>

2016-10-28 21:41 GMT+02:00 Zachary Bedell <zbedell@nycourts.gov<mailto:zbedell@nycourts.gov>
<mailto:
z
bedell@nycourts.gov<mailto:bedell@nycourts.gov>><mailto:z
bedell@nycourts.gov<mailto:bedell@nycourts.gov><mailto:bedell@nycourts.gov>>>:

Would the stats show up in a different part of the tree?  I don't see
the
datasources entry at all under openejb.management.

I can find DataSourceFactory objects under:

Catalina:class=org.apache.openejb.resource.jdbc.
DataSourceFactory,name="
EARNAME/POOLNAME",resourcetype=Global,type=Resource


The DataSource's themselves show up in every WAR they're injected
into:

Catalina:type=DataSource,host=localhost,context=/WARNAME,
class=javax.sql.DataSource,name="openejb/Resource/POOLNAME"


Neither the DataSource nor DataSourceFactory expose anything that
looks
like active or max connections.

We're trying pretty hard to keep everything related to the app in its
own
EAR rather than globally in tomee.xml.  So far the only thing we have
in
tomee.xml is a <Deployments dir="X"/> entry where we're dropping the
EAR's.

-Zac

On Oct 28, 2016, at 15:28, Romain Manni-Bucau <rmannibucau@gmail.com<mailto:rmannibucau@gmail.com>
<mailto:rmannibucau@gmail.com>
<mailto:rmannibucau@gmail.com>>
wrote:

they are, name is just prefixed with app name


Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://blog-rmannibucau.rhcloud.com<https://blog-
rmannibucau.rhcloud.com/<http://rmannibucau.rhcloud.com/>>> | Old Blog
<http://rmannibucau.wordpress.com<http://rmannibucau.wordpress.com/>>
|
Github <https://github.com/
rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory
<https://javaeefactory-rmannibucau.rhcloud.com<https:
//javaeefactory-rmannibucau.rhcloud.com/<http://javaeefactory-rmannibucau.rhcloud.com/>>>

2016-10-28 21:14 GMT+02:00 Adam Cornett <adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><
mailto:
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com>><mailto:
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><mailto:adam.cornett@gmail.com>>>:

Perhaps, my datasources are defined at the container level
(conf/tomee.xml). Perhaps the app level data sources are not
registered
in
JMX.
Romain or another dev would need to speak to that.

On Fri, Oct 28, 2016 at 2:29 PM, Zachary Bedell <zbedell@nycourts.gov<mailto:zbedell@nycourts.gov>
<mailto:zbedell@nycourts.gov>
<mailto:zbedell@nycourts.gov>>
wrote:

Yes to TomEE 7.0.1, but I don't see a datasources entry under
openejb.management in jconsole.  I've got Invocations, JAX-RS, JAX-WS,
Pool, TransactionManager, and containers.

Guessing maybe the way I define the datasources may be relevant?
They're
in a resources.xml file inside an EAR.  Defined like:

<Resource id="ucmsTXPool" type="DataSource">
JdbcDriver oracle.jdbc.xa.client.OracleXADataSource
JdbcUrl ${ucmsTXPool.JdbcUrl}
User ${ucmsTXPool.User}
UserName ${ucmsTXPool.User}
VaultPassword ${ucmsTXPool.Password}
InitialSize ${ucmsTXPool.InitialSize}
MaxActive ${ucmsTXPool.MaxActive}
MaxIdle 10
MaxWait 15000
ValidationQuery "SELECT sysdate FROM DUAL"
TestOnBorrow true
TestOnReturn true
TestWhileIdle true
TimeBetweenEvictionRunsMillis 60000
AccessToUnderlyingConnectionAllowed true
JtaManaged true
</Resource>

The ${...} are loaded from a config repository via a Listener in
server.xml.  The VaultPassword is handled via custom PasswordCipher
registered via META-INF/org.apache.openejb.cipher.PasswordCipher in a
jar
in server/lib.

-Zac

On Oct 28, 2016, at 14:09, Adam Cornett <adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><
mailto:
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com>><mailto:
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><mailto:adam.cornett@gmail.com>>>
wrote:

Sure:
openejb.management/datasources/[DSNAME] should have attributes such
as:
Size, Idle, WaitCount, etc.

There should be an object for each datasource defined in your
tomee.xml
file.  I'm assuming you're on TomEE 7?

On Fri, Oct 28, 2016 at 2:03 PM, Zachary Bedell <zbedell@nycourts.gov<mailto:zbedell@nycourts.gov>
<mailto:zbedell@nycourts.gov>
<mailto:zbedell@nycourts.gov>

wrote:

Adam, your image didn't come through the list, at least for me.  Can
you
give me a textual path to where you're looking?


The only thing listed as openejb under mbeans is
"openejb.management."
There's nothing under there that matches the names of any of my
pools.
The
only matches for "datasource" I see are the containers for the
stateless,
stateful, singleton, and managed beans.  The only pool related stuff
I
see
under there is for the EJB instances.

I see an "Active" attribute under the TransactionManager, but that's
not
per-pool, and I'm not sure that it represents all of the JDBC
connections
as opposed to an EJB container transaction.  We'd have EJB's that are
set
@TransactionAttribute(NOT_SUPPORTED) as well as single transactions
that
touch several connection pools (XA), so active transactions from the
container's point of view isn't granular enough to know if one of the
pools
is running low.

-Zac




On Oct 28, 2016, at 13:30, Adam Cornett <adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><
mailto:
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com>><mailto:
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><mailto:adam.cornett@gmail.com>><
mailto:
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><mailto:adam.cornett@gmail.com><mailto:
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com>>>> wrote:

Here is a screenshot out of jconsole showing the location and
attributes:


On Fri, Oct 28, 2016 at 1:21 PM, Romain Manni-Bucau <
rmannibucau@gmail.com<mailto:rmannibucau@gmail.com><mailto:rmannibucau@gmail.com><mailto:r
mannibucau@gmail.com<mailto:mannibucau@gmail.com>>
<mailto:rmannibucau@gmail.com>> wrote:
Hi Zachary,

2016-10-28 19:18 GMT+02:00 Zachary Bedell <zbedell@nycourts.gov<mailto:zbedell@nycourts.gov>
<mailto:
z
bedell@nycourts.gov<mailto:bedell@nycourts.gov>>
<mailto:
z
bedell@nycourts.gov<mailto:bedell@nycourts.gov><mailto:bedell@nycourts.gov><mailto:bedel
l@nycourts.gov<mailto:l@nycourts.gov>
:

Good afternoon,

I'm working on a monitoring agent for TomEE to plugin to an in-house
package we use.  One of the most common failures in our current app
server
(JBoss) usually manifests as expended JDBC connection pools, so we
alert
as
pools approach full to get some kind of warning of impending doom
when
the
database can't keep up.

I need to get a handle on TomEE's datasource pools to check their
maximum
size & current active connection count programatically.  I've tried
searching through both JMX (how we do it in JBoss) and the JNDI
tree.
I
can find the javax.sql.DataSource in both trees as well as a
reference
to
org.apache.openejb.resource.jdbc.DataSourceFactory in JMX.  It
doesn't
appear any of the objects I've been able to find expose a count of
active
connections nor what the configured maximum is.


in openejb MBeans it should be there


Is there a way to get the count of active connections & the
configured
max
for a named connection pool (I can find the names via JMX or JNDI)
or
for
all connection pools?



Thanks in advance,
Zac Bedell





--
Adam Cornett
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><mailto:
adam.cornett@gmail.com><mailto:
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com>>
(678) 296-1150




--
Adam Cornett
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><mailto:
adam.cornett@gmail.com>
(678) 296-1150




--
Adam Cornett
adam.cornett@gmail.com<mailto:adam.cornett@gmail.com><mailto:
adam.cornett@gmail.com>
(678) 296-1150








Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message