hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: HDFS data locality
Date Tue, 17 Nov 2009 22:49:36 GMT
Hi,

The master currently doesn't do anything specifically to assure the
locality. Maybe a future version will do so?

During the continuous operation of a regionserver, it will do
compactions which is essentially rewriting all the data. The
compactions create data locality. Generally on my prod cluster regions
get into a stable assignment and stay there for months on end.  If you
are not taking your cluster up and down multiple times a day, the
locality becomes very strong.

-ryan

On Tue, Nov 17, 2009 at 2:43 PM, Igor Katkov <ikatkov@gmail.com> wrote:
> Hi,
>
> When HMaster assigns regions to region servers does it try to ensure that
> these files will be located on the same host in HDFS? It does not, does not
> it?
> So most likely HBase RegionServers are very chatty over the network, reading
> and writing from/to the HDFS daemons on other nodes.
>
> Is there a way to improve it? To make RegionServer mostly talk to the local
> DataNode only?
>

Mime
View raw message