hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans" <jdcry...@gmail.com>
Subject Re: Building an Hadoop & Hbase Cluster
Date Sun, 14 Dec 2008 16:32:02 GMT
Yossi,

I will answer according to what I think are the best practices but this will
require many machines and for a little cluster, as you will see, it may be
hard to apply.

1 and 2. You should have X RS for X DN on X machines. This is to benefit
from the locality of data (when it does happen).

3. No, that would make 2 SPOFs at the same place. (but we are currently
integrating Zookeeper in HBase so the HMaster SPOF will be fixed hopefully
in 0.20)

4. IIRC, the SNN has the same memory requirements that the NN and that can
get very big depending on your number of files. So I would say that you
shouldn't do that especially because a RS will also live on that node.

I also recommend looking at Jeff Dean's slides presented at the
Data-Intensive Computing Symposium since what you want to do is similar to
what Google already does: http://research.yahoo.com/files/6DeanGoogle.pdf

J-D

On Sun, Dec 14, 2008 at 3:18 AM, Yossi Ittach <yossale@gmail.com> wrote:

> Hi
>
> I want to build a little cluster of Hbase & Hadoop. Starngely , I couldn't
> find any recommendations on the web.
> If you could share your experience , it would be great , specially in
> concern with these questions :
>
> 1. How many RegionServers I need for X DataNodes?
> 2. Can I (and should I?) run a RegionServer and a DataNode on the same
> machine?
> 3. Can I run the NameNode and the HBaseMaster on the same machine?
> 4. Can I run a Secondary Name Node and a DataNode on the same machine?
>
> Thanks!
>
>
> Vale et me ama
> Yossi
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message