qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Wall (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (QPID-6994) [Java Broker] AMQP connection close might fail to delete temporary queue after close of VirtualHost
Date Tue, 12 Jan 2016 17:11:39 GMT

     [ https://issues.apache.org/jira/browse/QPID-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Keith Wall updated QPID-6994:
-----------------------------
    Description: 
With current implementation of AMQPConnection close we invoke VirtualHost#deregisterConnection
first before we invoke AbstractAMQPConnection#performDeleteTasks. As result, Connection configured
object associated with VirtualHost is unregistered before connection deletion task is executed.
If VH is closing at the same time  when connection is closed, it might happen that after call
VirtualHost#deregisterConnection the VH can close immediately and the following call to AbstractAMQPConnection#performDeleteTasks
would fail because queue deletion task cannon be performed on already closed virtual host.

The problem can manifest either a) during Broker shutdown, b) during a virtual host node state
change (e.g. an operator stopping the virtual host node from the console).  Stopping the virtual
host itself, or a change in mastership during a HA election cannot cause this problem.


The exception like the one below is reported in this case:
{noformat}
2016-01-10 21:45:08,765         ERROR [VirtualHostNode-test-Config] o.a.q.s.c.u.TaskExecutorImpl
Task executor VirtualHostNode-test-Config is not in ACTIVE state, unable to execute : Task['set
desired state' on 'Binding{bindingKey=testTopic, exchange=TopicExchange[amq.topic], queue=tmp_127_0_0_1_51683_1,
id= 0c34946e-5e28-4d8c-a84e-50fb77deee03 }' with arguments 'DELETED'] 

at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.checkState(TaskExecutorImpl.java:225)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.submitWrappedTask(TaskExecutorImpl.java:147)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.submit(TaskExecutorImpl.java:142)
	at org.apache.qpid.server.model.AbstractConfiguredObject.doOnConfigThread(AbstractConfiguredObject.java:512)
	at org.apache.qpid.server.model.AbstractConfiguredObject.setDesiredState(AbstractConfiguredObject.java:1385)
	at org.apache.qpid.server.model.AbstractConfiguredObject.deleteAsync(AbstractConfiguredObject.java:2001)
	at org.apache.qpid.server.queue.AbstractQueue.deleteAndReturnCountAsync(AbstractQueue.java:1932)
	at org.apache.qpid.server.queue.AbstractQueue.doDelete(AbstractQueue.java:3078)
	at sun.reflect.GeneratedMethodAccessor238.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.qpid.server.model.AbstractConfiguredObject.attainState(AbstractConfiguredObject.java:1239)
	at org.apache.qpid.server.model.AbstractConfiguredObject.access$1900(AbstractConfiguredObject.java:78)
	at org.apache.qpid.server.model.AbstractConfiguredObject$15.execute(AbstractConfiguredObject.java:1427)
	at org.apache.qpid.server.model.AbstractConfiguredObject$15.execute(AbstractConfiguredObject.java:1386)
	at org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:520)
	at org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:513)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$TaskLoggingWrapper.execute(TaskExecutorImpl.java:270)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper$1.run(TaskExecutorImpl.java:342)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:356)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper.call(TaskExecutorImpl.java:335)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
{noformat}

  was:
With current implementation of AMQPConnection close we invoke VirtualHost#deregisterConnection
first before we invoke AbstractAMQPConnection#performDeleteTasks. As result, Connection configured
object associated with VirtualHost is unregistered before connection deletion task is executed.
If VH is closing at the same time when connection is closed, it might happen that after call
VirtualHost#deregisterConnection the VH can close immediately and the following call to AbstractAMQPConnection#performDeleteTasks
would fail because queue deletion task cannon be performed on already closed virtual host.

The exception like the one below is reported in this case:
{noformat}
2016-01-10 21:45:08,765         ERROR [VirtualHostNode-test-Config] o.a.q.s.c.u.TaskExecutorImpl
Task executor VirtualHostNode-test-Config is not in ACTIVE state, unable to execute : Task['set
desired state' on 'Binding{bindingKey=testTopic, exchange=TopicExchange[amq.topic], queue=tmp_127_0_0_1_51683_1,
id= 0c34946e-5e28-4d8c-a84e-50fb77deee03 }' with arguments 'DELETED'] 

at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.checkState(TaskExecutorImpl.java:225)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.submitWrappedTask(TaskExecutorImpl.java:147)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.submit(TaskExecutorImpl.java:142)
	at org.apache.qpid.server.model.AbstractConfiguredObject.doOnConfigThread(AbstractConfiguredObject.java:512)
	at org.apache.qpid.server.model.AbstractConfiguredObject.setDesiredState(AbstractConfiguredObject.java:1385)
	at org.apache.qpid.server.model.AbstractConfiguredObject.deleteAsync(AbstractConfiguredObject.java:2001)
	at org.apache.qpid.server.queue.AbstractQueue.deleteAndReturnCountAsync(AbstractQueue.java:1932)
	at org.apache.qpid.server.queue.AbstractQueue.doDelete(AbstractQueue.java:3078)
	at sun.reflect.GeneratedMethodAccessor238.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.qpid.server.model.AbstractConfiguredObject.attainState(AbstractConfiguredObject.java:1239)
	at org.apache.qpid.server.model.AbstractConfiguredObject.access$1900(AbstractConfiguredObject.java:78)
	at org.apache.qpid.server.model.AbstractConfiguredObject$15.execute(AbstractConfiguredObject.java:1427)
	at org.apache.qpid.server.model.AbstractConfiguredObject$15.execute(AbstractConfiguredObject.java:1386)
	at org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:520)
	at org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:513)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$TaskLoggingWrapper.execute(TaskExecutorImpl.java:270)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper$1.run(TaskExecutorImpl.java:342)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:356)
	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper.call(TaskExecutorImpl.java:335)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
{noformat}


> [Java Broker] AMQP connection close might fail to delete temporary queue after close
of VirtualHost
> ---------------------------------------------------------------------------------------------------
>
>                 Key: QPID-6994
>                 URL: https://issues.apache.org/jira/browse/QPID-6994
>             Project: Qpid
>          Issue Type: Bug
>          Components: Java Broker
>    Affects Versions: qpid-java-6.0, qpid-java-6.1
>            Reporter: Alex Rudyy
>
> With current implementation of AMQPConnection close we invoke VirtualHost#deregisterConnection
first before we invoke AbstractAMQPConnection#performDeleteTasks. As result, Connection configured
object associated with VirtualHost is unregistered before connection deletion task is executed.
If VH is closing at the same time  when connection is closed, it might happen that after call
VirtualHost#deregisterConnection the VH can close immediately and the following call to AbstractAMQPConnection#performDeleteTasks
would fail because queue deletion task cannon be performed on already closed virtual host.
> The problem can manifest either a) during Broker shutdown, b) during a virtual host node
state change (e.g. an operator stopping the virtual host node from the console).  Stopping
the virtual host itself, or a change in mastership during a HA election cannot cause this
problem.
> The exception like the one below is reported in this case:
> {noformat}
> 2016-01-10 21:45:08,765         ERROR [VirtualHostNode-test-Config] o.a.q.s.c.u.TaskExecutorImpl
Task executor VirtualHostNode-test-Config is not in ACTIVE state, unable to execute : Task['set
desired state' on 'Binding{bindingKey=testTopic, exchange=TopicExchange[amq.topic], queue=tmp_127_0_0_1_51683_1,
id= 0c34946e-5e28-4d8c-a84e-50fb77deee03 }' with arguments 'DELETED'] 
> at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.checkState(TaskExecutorImpl.java:225)
> 	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.submitWrappedTask(TaskExecutorImpl.java:147)
> 	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.submit(TaskExecutorImpl.java:142)
> 	at org.apache.qpid.server.model.AbstractConfiguredObject.doOnConfigThread(AbstractConfiguredObject.java:512)
> 	at org.apache.qpid.server.model.AbstractConfiguredObject.setDesiredState(AbstractConfiguredObject.java:1385)
> 	at org.apache.qpid.server.model.AbstractConfiguredObject.deleteAsync(AbstractConfiguredObject.java:2001)
> 	at org.apache.qpid.server.queue.AbstractQueue.deleteAndReturnCountAsync(AbstractQueue.java:1932)
> 	at org.apache.qpid.server.queue.AbstractQueue.doDelete(AbstractQueue.java:3078)
> 	at sun.reflect.GeneratedMethodAccessor238.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.qpid.server.model.AbstractConfiguredObject.attainState(AbstractConfiguredObject.java:1239)
> 	at org.apache.qpid.server.model.AbstractConfiguredObject.access$1900(AbstractConfiguredObject.java:78)
> 	at org.apache.qpid.server.model.AbstractConfiguredObject$15.execute(AbstractConfiguredObject.java:1427)
> 	at org.apache.qpid.server.model.AbstractConfiguredObject$15.execute(AbstractConfiguredObject.java:1386)
> 	at org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:520)
> 	at org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:513)
> 	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$TaskLoggingWrapper.execute(TaskExecutorImpl.java:270)
> 	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper$1.run(TaskExecutorImpl.java:342)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:356)
> 	at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper.call(TaskExecutorImpl.java:335)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message