qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Godfrey (Jira)" <j...@apache.org>
Subject [jira] [Commented] (QPID-8401) [Broker-J] Broker dies when DB connection is lost
Date Mon, 13 Jan 2020 15:31:00 GMT

    [ https://issues.apache.org/jira/browse/QPID-8401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17014413#comment-17014413
] 

Robert Godfrey commented on QPID-8401:
--------------------------------------

{quote}Now regarding the current behavior of the broker, note that when the broker dies this
way I end up with a message that stays acquired forever (not by any client consumer but by
the housekeeping task itself). When I restart the broker this message never gets deleted even
though it expired and I can't consume it either... Shouldn't the message be released when
rolling back the failed dequeue operation? [https://github.com/apache/qpid-broker-j/blob/master/broker-core/src/main/java/org/apache/qpid/server/queue/AbstractQueue.java#L1852]
{quote}
So acquisition should not be persisted, so when the broker dies and then restarts the message
should not be acquired.  In terms of the onRollback in the code quoted - that logic is for
when a transaction is manually rolled back by a client (not applicable in this case), it is
not designed for handling the case where the broker initiates rollback - in this case the
expected behaviour is that the broker (really should be the virtual host) dies.

> [Broker-J] Broker dies when DB connection is lost
> -------------------------------------------------
>
>                 Key: QPID-8401
>                 URL: https://issues.apache.org/jira/browse/QPID-8401
>             Project: Qpid
>          Issue Type: Bug
>          Components: Broker-J
>    Affects Versions: qpid-java-broker-7.1.6
>            Reporter: Olivier VERMEULEN
>            Priority: Critical
>
> When using a JDBC message store, if the housekeeping task is triggered while the DB connection
is lost (DB down or network problem) then the Broker dies with the stack below.
> This happens when a message expires and the housekeeping task tries to delete it from
the store while the DB is not accessible. In this case a StoreException is thrown but this
exception is not catched by the Housekeeping task which is only catching ConnectionScopedRuntimeExceptions.
>  
> 2019-12-12 16:22:40,671 ERROR [virtualhost-default-pool-3] (o.a.q.s.Main) - Uncaught
exception, shutting down.
> org.apache.qpid.server.store.StoreException: java.sql.SQLException: JZ006: Caught IOException:
com.sybase.jdbc4.jdbc.SybConnectionDeadException: JZ0C0: Connection is already closed.
>  at org.apache.qpid.server.store.jdbc.AbstractJDBCMessageStore$JDBCTransaction.<init>(AbstractJDBCMessageStore.java:1153)
>  at org.apache.qpid.server.store.jdbc.GenericAbstractJDBCMessageStore$RecordedJDBCTransaction.<init>(GenericAbstractJDBCMessageStore.java:122)
>  at org.apache.qpid.server.store.jdbc.GenericAbstractJDBCMessageStore$RecordedJDBCTransaction.<init>(GenericAbstractJDBCMessageStore.java:118)
>  at org.apache.qpid.server.store.jdbc.GenericAbstractJDBCMessageStore.newTransaction(GenericAbstractJDBCMessageStore.java:114)
>  at org.apache.qpid.server.txn.AutoCommitTransaction.dequeue(AutoCommitTransaction.java:87)
>  at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1780)
>  at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1775)
>  at org.apache.qpid.server.queue.AbstractQueue.deleteEntry(AbstractQueue.java:1819)
>  at org.apache.qpid.server.queue.AbstractQueue.expireEntry(AbstractQueue.java:2354)
>  at org.apache.qpid.server.queue.AbstractQueue.getNextAvailableEntry(AbstractQueue.java:2236)
>  at org.apache.qpid.server.queue.AbstractQueue.access$1800(AbstractQueue.java:131)
>  at org.apache.qpid.server.queue.AbstractQueue$AdvanceConsumersTask.execute(AbstractQueue.java:3712)
>  at org.apache.qpid.server.virtualhost.HouseKeepingTask$1.run(HouseKeepingTask.java:56)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at org.apache.qpid.server.virtualhost.HouseKeepingTask.run(HouseKeepingTask.java:51)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  at org.apache.qpid.server.bytebuffer.QpidByteBufferFactory.lambda$null$0(QpidByteBufferFactory.java:464)
>  at java.lang.Thread.run(Thread.java:748)
> Caused by: java.sql.SQLException: JZ006: Caught IOException: com.sybase.jdbc4.jdbc.SybConnectionDeadException:
JZ0C0: Connection is already closed.
>  at com.sybase.jdbc4.jdbc.ErrorMessage.createIOEKilledConnEx(ErrorMessage.java:1155)
>  at com.sybase.jdbc4.jdbc.ErrorMessage.raiseErrorCheckDead(ErrorMessage.java:1194)
>  at com.sybase.jdbc4.tds.Tds.handleIOE(Tds.java:5250)
>  at com.sybase.jdbc4.tds.Tds.handleIOE(Tds.java:5195)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message