tomee-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matej <gma...@gmail.com>
Subject Re: ActiveMQ failover
Date Sat, 11 Oct 2014 13:45:51 GMT
I have done some analyzing. It seems ActiveMQ creates some illegal state.
And then does throws an illegalstateexception, after that, connections are
in state close.

Don't know if ActiveMQ libs or, the geronimo connection pool doesn't remove
the closed connections. I will try with keepAlive tcp flag. Maybe this will
help.

If anyone would like to see. I appending logs.

BR

Matej

2014-10-10 17:33 GMT+02:00 Matej <gmatej@gmail.com>:

> Hello Romain and Andy.
>
> Thanks for helping.
>
> We have only one ActiveMQ  5.10 server not embedded. So actually we
> probably want only retry. For retry I have to write the hostname twice in
> Tomee (1.7 - Java 8)?
>
> failover:(tcp://10.1.1.1:61616,tcp://10.1.1.1:61616)
>
> Actually when restarting Active MQ, the reconnection procedure works
> nicely. While  listing the host only once in failover brackets.
>
>
> This is how we send messages:
>
>   protected void send(ConnectionFactory connectionFactory, Queue etaQueue, String text,
int deliveryMode) throws JMSException {
>
>
>         Connection connection = null;
>         Session session = null;
>         try {
>             connection = connectionFactory.createConnection();
>             connection.start();
>             session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>
>             MessageProducer producer = session.createProducer(etaQueue);
>             producer.setDeliveryMode(deliveryMode);
>             TextMessage message = session.createTextMessage(text);
>             producer.send(message);
>         } finally {
>             if (session != null) {
>                 session.close();
>             }
>             if (connection != null) {
>                 connection.close();
>             }
>         }
>     }
>
> Where this class is extended, with the EtaUpdateReqHandler   Statelles bean  that has
the @Resource anootation for connection factory, queue,...
>
>
> But still, after some time (1-2 days) in prod we get exceptions like that:
>
> INFO: Removing non-required WorkContextHandler with no context: org.apache.geronimo.connector.work.HintsContextHandler@6c4f8558
> Oct 09, 2014 5:18:42 PM org.apache.geronimo.connector.outbound.SinglePoolConnectionInterceptor
doRemove
> INFO: Removing ManagedConnectionInfo: org.apache.geronimo.connector.outbound.ManagedConnectionInfo@287fc15.
mc: org.apache.activemq.ra.ActiveMQManagedConnection@4c511a08] from pool org.apache.geronimo.connector.outbound.SinglePoolConnectionInterceptor@27e7e9b4
> Oct 09, 2014 5:18:42 PM org.apache.activemq.TransactionContext commit
> WARNING: commit of: XID:[1197822575,globalId=ffffffebffffffdc630000047544d4944000000000000000000000000000000000000000000000000000,branchId=2000000047544d4944000000000000000000000000000000000000000000000000000]
failed with: javax.jms.JMSException: Stopped.
> javax.jms.JMSException: Stopped.
>  at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:54)
>  at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1417)
>  at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1522)
>  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>  at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Stopped.
>  at org.apache.activemq.transport.ResponseCorrelator.stop(ResponseCorrelator.java:131)
>  at org.apache.activemq.util.ServiceSupport.dispose(ServiceSupport.java:43)
>  Oct 09, 2014 5:18:42 PM org.apache.geronimo.transaction.manager.CommitTask run
> SEVERE: Unexpected exception committing org.apache.geronimo.transaction.manager.WrapperNamedXAResource@5a226dfb;
continuing to commit other RMs
> javax.transaction.xa.XAException: Stopped.
>  at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:805)
>  at org.apache.activemq.TransactionContext.commit(TransactionContext.java:594)
>  at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92)
>
>
> I would be hard for me to produce some sample, I'd had to simulate 200-400 request/s.
Without this load everything is ok.
>
> Oh, and restarting ActiveMQ usually doesn't solve things.
>
>
> BR
>
> MAtej
>
>
>
>
> 2014-10-10 14:42 GMT+02:00 Andy Gumbrecht <agumbrecht@tomitribe.com>:
>
>> Hi Matej,
>>
>> You need to specify at least two hosts to the failover for it to work -
>> It works like a retry when you use the same host.
>>
>> failover:(tcp://10.1.1.1:61616,tcp://10.1.1.1:61616)
>>
>> However, the error you are getting is possibly a design  issue in
>> EtaUpdateReqHandler - Never cache the connection locally in an instance
>> variable. Use:
>>
>> void method(){
>> Connection con = null;
>> try{
>> con = getNewConEveryTime();
>> }finally{
>> if(null != con){
>> con.close();
>> }
>> }
>> }
>>
>> Andy.
>>
>> On 08/10/2014 14:57, Matej wrote:
>>
>>> Hello all.
>>>
>>> Anyone with experience with TomEE and ActiveMQ in production. We seem to
>>> have problems with connections hanging, some poolling problem. Tomee s
>>> connection factory returns to the Producer already closed connections.
>>>
>>> We have tried with
>>>
>>>          BrokerXmlConfig =
>>>          ServerUrl       =  failover:(tcp://10.1.1.1:61616)
>>>
>>> This does not seem to be working. Anything else we can do?
>>>
>>> BR
>>>
>>> Matej
>>>
>>>
>>> BR
>>>
>>> Matej
>>>
>>>
>>
>> --
>>   Andy Gumbrecht
>>   https://twitter.com/AndyGeeDe
>>   http://www.tomitribe.com
>>
>>
>

Mime
View raw message