hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Smith <cbsm...@gmail.com>
Subject Re: Hdfs over RDMA
Date Tue, 22 Feb 2011 05:15:13 GMT
On Fri, Feb 18, 2011 at 10:32 AM, Rajat Sharma <sh.rajat7@gmail.com> wrote:
> Hi Guys
> I am trying to work on HDFS to improve its performance by adding RDMA
> functionality to its code. But i cannot find any sort of documentation or
> help regarding this topic except some information about Socket Direct
> Protocol or Allocated buffers. since RDMA is the premier protocol which is
> highly preferred for Distributed File Systems , why is it not preferred in
> case of HDFS or what are the factors which makes it tough. Any suggestions
> or papers or ideas regarding the ways we can get some way to work on it. I
> am using OFED Distribution , RDMA for this.

HDFS really serves a different purpose from other distributed
filesystems. I'd argue that RDMA is a premature optimization that
would introduce a lot of complexity in to the code base. If you are
using it right, much of your data traffic with HDFS doesn't go over
the network at all.


View raw message