qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Wall (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (QPID-6998) [Java Broker, BDBStore, HA] After HA Environment transition from MASTER into DETACHED/WAITING/UNKNOWN and back into MASTER the calls to Database instance created whilst Environment was Master result in IllegalStateException
Date Sat, 23 Jan 2016 20:11:39 GMT

    [ https://issues.apache.org/jira/browse/QPID-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113936#comment-15113936
] 

Keith Wall commented on QPID-6998:
----------------------------------

This is actual a duplicate of QPID-6997.  Broker 3's previous environment restart at 2016-01-14
14:17:50,485 failed.  The  restart closed the databases successfully, but then a database
(MESSAGE_CONTENT) was reopened and *re-added to the cache*.  The environment then failed to
close (reporting the open database and open cursor). The environment was then recreated successfully,
but the database associated with the old environment remained in the cache.

It is that that caused the exception reported above. The fix for QPID-6997 will no longer
allow this to happen.

Closing as Invalid.

> [Java Broker, BDBStore, HA] After HA Environment transition from MASTER into DETACHED/WAITING/UNKNOWN
 and back into MASTER the calls to Database instance created whilst Environment was Master
result in IllegalStateException
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: QPID-6998
>                 URL: https://issues.apache.org/jira/browse/QPID-6998
>             Project: Qpid
>          Issue Type: Bug
>          Components: Java Broker
>    Affects Versions: 0.32, qpid-java-6.0, qpid-java-6.1
>            Reporter: Alex Rudyy
>         Attachments: TEST-org.apache.qpid.server.store.berkeleydb.replication.MultiNodeTest.testInFlightTransactionsWhilstMajorityIsLost.txt.STALE_DB
>
>
> After HA Environment transition from MASTER into DETACHED/WAITING state and back into
MASTER the calls to Database instance created whilst Environment was Master result in IllegalStateException
followed by immediate Broker shutdown
> The following was reported in the broker logs:
> {noformat}
> ########################################################################
> #
> # Unhandled Exception org.apache.qpid.server.store.StoreException: Error getting AMQMessage
with id 100001 to database: Attempt to use non-open Environment object(). in Thread IO-/127.0.0.1:38948
> #
> # Exiting
> #
> ########################################################################
> org.apache.qpid.server.store.StoreException: Error getting AMQMessage with id 100001
to database: Attempt to use non-open Environment object().
>  at org.apache.qpid.server.store.berkeleydb.replication.ReplicatedEnvironmentFacade.handleDatabaseException(ReplicatedEnvironmentFacade.java:457)
>  at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.getAllContent(AbstractBDBMessageStore.java:407)
>  at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContentAsByteBuffer(AbstractBDBMessageStore.java:1115)
>  at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContent(AbstractBDBMessageStore.java:1130)
>  at org.apache.qpid.server.message.AbstractServerMessageImpl.getContent(AbstractServerMessageImpl.java:177)
>  at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$MessageContentSourceBody.writePayload(ProtocolOutputConverterImpl.java:261)
>  at org.apache.qpid.framing.AMQFrame.writePayload(AMQFrame.java:69)
>  at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$CompositeAMQBodyBlock.writePayload(ProtocolOutputConverterImpl.java:459)
>  at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8.writeFrame(AMQPConnection_0_8.java:395)
>  at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeFrame(ProtocolOutputConverterImpl.java:416)
>  at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDeliveryUnchanged(ProtocolOutputConverterImpl.java:215)
>  at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:138)
>  at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:95)
>  at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeDeliver(ProtocolOutputConverterImpl.java:72)
>  at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$WriteDeliverMethod.deliverToClient(AMQPConnection_0_8.java:1366)
>  at org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8.sendToClient(ConsumerTarget_0_8.java:449)
>  at org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8$AckConsumer.doSend(ConsumerTarget_0_8.java:275)
> ERROR [IO-/127.0.0.1:38948] o.a.q.s.u.ServerScopedRuntimeException Error getting AMQMessage
with id 100001 to database: Attempt to use non-open Environment object().
> java.lang.IllegalStateException: Attempt to use non-open Environment object().
>         at com.sleepycat.je.Environment.checkHandleIsValid(Environment.java:2193) ~[je-5.0.104.jar:5.0.104]
>         at com.sleepycat.je.Database.checkEnv(Database.java:2148) ~[je-5.0.104.jar:5.0.104]
>         at com.sleepycat.je.Database.get(Database.java:1266) ~[je-5.0.104.jar:5.0.104]
>         at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.getAllContent(AbstractBDBMessageStore.java:382)
~[qpid-bdbstore-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContentAsByteBuffer(AbstractBDBMessageStore.java:1115)
[qpid-bdbstore-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContent(AbstractBDBMessageStore.java:1130)
[qpid-bdbstore-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.message.AbstractServerMessageImpl.getContent(AbstractServerMessageImpl.java:177)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$MessageContentSourceBody.writePayload(ProtocolOutputConverterImpl.java:261)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.framing.AMQFrame.writePayload(AMQFrame.java:69) [qpid-common-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$CompositeAMQBodyBlock.writePayload(ProtocolOutputConverterImpl.java:459)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8.writeFrame(AMQPConnection_0_8.java:395)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeFrame(ProtocolOutputConverterImpl.java:416)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDeliveryUnchanged(ProtocolOutputConverterImpl.java:215)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:138)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:95)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeDeliver(ProtocolOutputConverterImpl.java:72)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$WriteDeliverMethod.deliverToClient(AMQPConnection_0_8.java:1366)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8.sendToClient(ConsumerTarget_0_8.java:449)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8$AckConsumer.doSend(ConsumerTarget_0_8.java:275)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.consumer.AbstractConsumerTarget.sendNextMessage(AbstractConsumerTarget.java:219)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.consumer.AbstractConsumerTarget.processPending(AbstractConsumerTarget.java:68)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.AMQChannel.processPending(AMQChannel.java:3768)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$ProcessPendingIterator$1.run(AMQPConnection_0_8.java:1556)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.transport.NonBlockingConnection.processPending(NonBlockingConnection.java:330)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:247)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:108)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.processConnection(SelectorThread.java:499)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.run(SelectorThread.java:487)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:457)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_80]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_80]
>         at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> {noformat}
> It appears we need to re-create environment on transitions: MASTER->DETACHED/WAITING/UNKNOWN->MASTER



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message