qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ken Giusti (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (QPID-5637) Python client does not reset the Selector singleton when the process id changes.
Date Tue, 20 May 2014 18:20:39 GMT

     [ https://issues.apache.org/jira/browse/QPID-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ken Giusti updated QPID-5637:
-----------------------------

    Affects Version/s: 0.26
              Summary: Python client does not reset the Selector singleton when the process
id changes.  (was: qpid.messaging Issues With Forking)

> Python client does not reset the Selector singleton when the process id changes.
> --------------------------------------------------------------------------------
>
>                 Key: QPID-5637
>                 URL: https://issues.apache.org/jira/browse/QPID-5637
>             Project: Qpid
>          Issue Type: Bug
>          Components: Python Client
>    Affects Versions: 0.24, 0.26
>            Reporter: Brian Bouterse
>            Assignee: Ken Giusti
>            Priority: Blocker
>             Fix For: 0.27
>
>         Attachments: celery_worker_output.txt, pid_aware_selector.patch, tasks.py
>
>
> qpid.messaging has an issue with forking in the following situation.
> 1.  A parent Python process imports and uses qpid.messaging to connect to a a Qpid broker
> 2.  The parent process forks a child process
> 3.  The child process imports qpid.messaging and tries to connect to a Qpid broker.
> I expected to see the child process use qpid.messaging normally as it would if it weren't
forked in the way described above.  Instead, the server receives the opening of a TCP socket,
but the client never sends the AMQP protocol announcement.
> [Forking bring child descriptors with it|http://man7.org/linux/man-pages/man2/fork.2.html].
 I expected the file descriptors on the parent and the child to be the same, and to reference
the same socket, so I expect qpid.messaging to work without any modification.  Surprisingly,
it does not.
> There is at least one place where I do understand how this can be avoided.  One of the
issues is that the file descriptors registered by the Selector object inside of qpid.messaging
are stale after the fork.  The Selector object uses a singleton pattern to provide a reference
to the same Selector object no matter how many times you call it.  This selector object already
has registered file descriptors with the filesystem, which allow the selector to read/write
data in an I/O efficient manner.  See the attached [pid_aware_selector.patch] for an example
of this.
> The [pid_aware_selector.patch] does allow communication to flow, but  queue creation
and deletion sometimes fail in strange ways.  For instance, in the child process, code that
creates a queue, reads information about that queue next.  The queue was created, yet the
read says that the queue can't be found.  Very strange.  You can see those things fail using
the following example:
> 1.  clone our fork of kombu:       `git clone git@github.com:pulp/kombu.git`
> 2.  Change into the kombu folder     `cd kombu`
> 3.  Switch to the branch containing the qpid code:  `git checkout pulp-dep-3.0.15-with-qpid`
> 4.  Install kombu onto your system or virtualenv (I do it systemwide using sudo):   `sudo
python setup.py develop`
> 5.  install celery version 3.1.11.  I do it using pip.    `sudo pip install celery==3.1.11`
> 6.  Install qpid.messaging and qpidtoollibs.  One way I do it is systemwide using pip.
     `sudo pip install qpid-tools qpid-python`
> 7.  Start up qpidd (We've been testing with 0.24 and auth off).      `sudo -u qpidd qpidd
--auth=no`
> 8.  Put the attached file tasks.py into a directory
> 9.  Open two terminals and change their working directory to be the same as step 8.
> 10.  In one one terminal start the celery worker        `celery worker -A tasks --loglevel=INFO
-c1`
> 11.  In the other terminal dispatch 10 tasks             `python tasks.py`
> You should see exceptions raised similar to those in the attached file [celery_worker_output.txt]
> Note, that the code on the pulp-dep-3.0.15-with-qpid branch of kombu monkey patches qpid.messaging
with the selector patch referenced above, and also one or two other bugfix patches.  You can
see that [monkey patching done here|https://github.com/pulp/kombu/blob/pulp-dep-3.0.15-with-qpid/kombu/transport/qpid.py#L45].
 This should have no implications on this issue, but I want to be explicit about it.
> A potential fix:  Celery supports a callback after child processes are forked, allowing
the call to cleanup/reset exactly these types of things.  I could wire up that callback if
such a thing existed on qpid.messaging.  For testing purposes, you could put a call to this
cleanup method in the 'sometask' code before the call to controller.inspect().active_queues().
 This would be similar in timing to a post fork cleanup/reset call.
> Note: the original connection and associated senders/receivers/sessions are still in
use by the parent process, so calling close() is not the right thing to do either.  It's like
the connection needs to be forgotten, and the file descriptors unregistered from the child
process.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message