qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Stitcher <astitc...@redhat.com>
Subject Re: What are Qpid's plans for RDMA?
Date Mon, 29 Sep 2008 14:59:13 GMT

On Thu, 2008-09-25 at 20:20 -0400, gregory james marsh wrote:
> Hello,
> I'm with the Network-Based Computing Lab at Ohio State University.  Our
> lab primarily does research with the Message Passing Interface (MPI) over
> interconnects like InfiniBand and RDMA.  Recently we have started
> evaluating/testing Qpid with the TCP adaptations to InfiniBand (Sockets
> Direct Protocol [SDP] and IP over InfiniBand [IPoIB]).
> We would like to know more about where the Qpid project plans to go with
> RDMA.  Could anyone give me a brief overview and anticipated timeline of
> the planned functionality?

As of svn revision 697101 The RDMA support in trunk qpid is pretty
stable so if you are using a trunk version more recent than that you
have RDMA support.

The more recent changes have been performance/scalability related.

RDMA is implemented as 2 protocol plugins one for the broker side
(rdma.so) and one for the client side (rdmaconnector.so). In an
installed broker/client library the modules will be found/loaded
automatically. If compiling from scratch the GNU defaults are used and
the modules end up in /usr/local/lib/qpid/daemon
& /usr/local/lib/qpid/client.

However if you are running from within the development tree you'll need
--load-module <path to rdma.so> added to your broker command line. And
you'll need something like:

export QPID_LOAD_MODULE=<path to rdmaconnector.so>

before you run your client code to enable the client library to find the
rdma module.

If you are using the test client in the qpid source tree you can add -P
rdma (or --protocol rdma) to tell them to use rdma instead of tcp. As we
use rdmacm for connection management Infiniband connections need to be
running IPoIB with an assigned IP address as this is used by rdmacm.

>   I've noticed some RDMA-related code submissions to the C++ trunk
> this month by Andrew Stitcher.  I've been able to run the C++ broker
> listening for RDMA via the RdmaIOPlugin class (at least according to the
> broker log).  However, how does the client side work?  Does the
> RdmaConnector class integrate with the client's Connection class?  Also in
> src/tests I've noticed that "rdma" is an option for the ConnectionOptions
> class.  How will this class fit into the scheme of things?

In your own client code you can set ConnectionOptions.protocol to "rdma"
and provide it to Connection.open to get an rdma connection.

I hope this gives you the info you need.


View raw message