qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Bouterse (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (QPID-5637) qpid.messaging Issues With Forking
Date Fri, 25 Apr 2014 17:53:17 GMT

     [ https://issues.apache.org/jira/browse/QPID-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brian Bouterse updated QPID-5637:
---------------------------------

    Description: 
qpid.messaging has an issue with forking in the following situation.

1.  A parent Python process imports and uses qpid.messaging to connect to a a Qpid broker
2.  The parent process forks a child process
3.  The child process imports qpid.messaging and tries to connect to a Qpid broker.

I expected to see the child process use qpid.messaging normally as it would if it weren't
forked in the way described above.  Instead, the server receives the opening of a TCP socket,
but the client never sends the AMQP protocol announcement.

Forking bring child descriptors with it:    http://man7.org/linux/man-pages/man2/fork.2.html

One of the issues is that the file descriptors registered by the Selector object inside of
qpid.messaging are stale after the fork.  The Selector object uses a singleton pattern to
provide a reference to the same Selector object no matter how many times you call it.  This
selector object already has registered file descriptors with the filesystem, which allow the
selector to read/write data in an I/O efficient manner.

When forking occurs, the child process receives a copy of the Python running space (ie: the
Selector singleton), but the kernel is not registered with file descriptors that are available
in the child process' new file descriptor table.  This breaks usage of qpid.messaging in the
client process because when qpid.messaging relies on the epoll functionality of Selector,
Selector does not fulfill that expectation.

The solution is to have the class method that implements the singleton pattern on the Selector
object to be process aware, and implement the singleton pattern per process.

  was:
qpid.messaging has an issue with forking in the following situation.

1.  A parent Python process imports and uses qpid.messaging to connect to a a Qpid broker
2.  The parent process forks a child process
3.  The child process imports qpid.messaging and tries to connect to a Qpid broker.

I expected to see the child process use qpid.messaging normally as it would if it weren't
forked in the way described above.  Instead, the server receives the opening of a TCP socket,
but the client never sends the AMQP protocol announcement.

One of the issues is that the file descriptors registered by the Selector object inside of
qpid.messaging are stale after the fork.  The Selector object uses a singleton pattern to
provide a reference to the same Selector object no matter how many times you call it.  This
selector object already has registered file descriptors with the filesystem, which allow the
selector to read/write data in an I/O efficient manner.

When forking occurs, the child process receives a copy of the Python running space (ie: the
Selector singleton), but the kernel is not registered with file descriptors that are available
in the child process' new file descriptor table.  This breaks usage of qpid.messaging in the
client process because when qpid.messaging relies on the epoll functionality of Selector,
Selector does not fulfill that expectation.

The solution is to have the class method that implements the singleton pattern on the Selector
object to be process aware, and implement the singleton pattern per process.


> qpid.messaging Issues With Forking
> ----------------------------------
>
>                 Key: QPID-5637
>                 URL: https://issues.apache.org/jira/browse/QPID-5637
>             Project: Qpid
>          Issue Type: Bug
>          Components: Python Client
>    Affects Versions: 0.24
>            Reporter: Brian Bouterse
>             Fix For: 0.18, 0.22, 0.24, 0.26, 0.27
>
>         Attachments: 0001-Fix-for-QPID-5637.patch
>
>
> qpid.messaging has an issue with forking in the following situation.
> 1.  A parent Python process imports and uses qpid.messaging to connect to a a Qpid broker
> 2.  The parent process forks a child process
> 3.  The child process imports qpid.messaging and tries to connect to a Qpid broker.
> I expected to see the child process use qpid.messaging normally as it would if it weren't
forked in the way described above.  Instead, the server receives the opening of a TCP socket,
but the client never sends the AMQP protocol announcement.
> Forking bring child descriptors with it:    http://man7.org/linux/man-pages/man2/fork.2.html
> One of the issues is that the file descriptors registered by the Selector object inside
of qpid.messaging are stale after the fork.  The Selector object uses a singleton pattern
to provide a reference to the same Selector object no matter how many times you call it. 
This selector object already has registered file descriptors with the filesystem, which allow
the selector to read/write data in an I/O efficient manner.
> When forking occurs, the child process receives a copy of the Python running space (ie:
the Selector singleton), but the kernel is not registered with file descriptors that are available
in the child process' new file descriptor table.  This breaks usage of qpid.messaging in the
client process because when qpid.messaging relies on the epoll functionality of Selector,
Selector does not fulfill that expectation.
> The solution is to have the class method that implements the singleton pattern on the
Selector object to be process aware, and implement the singleton pattern per process.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message