hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andre Reiter <a.rei...@web.de>
Subject Re: hadoop / hbase /zookeeper architecture for best performance
Date Tue, 21 Jun 2011 11:31:51 GMT
Hi Mingjian,

so if i understand it right, the region servers should get as much memory as possible, correct?
at the moment our situation: the default amount of 1000 MB is used for the heap size (HBASE_HEAPSIZE)
 on region servers

so if we ran namenode, tasktracker and regionserver on the same hardware, how should the memory
distribution look like?
lets say for example 60% of memory for regionserver, 20% for namenode and 20% for tasktracker?
is that a proper configuration?

you use 16GB for the region server, is that an amound we should go for?

thanks iin advance

schrieb Mingjian Deng:
> Hi andre:
>      I think datanode and regionserver is in the same node is better. Because
> it will useful for locality.
>      zookeeper is lightly, so you can share them in your cluster with other
> processes. But in my opinion, you can running them in an extra cluster if
> there are more than 1 hbase cluster use the same zookeeper.
>      In our cluster, we have 43 nodes with 3 clusters. Two of them have 10
> regionservers and datanodes nodes and another has 20 regionservers and
> datanodes. And all the 3 clusters use 1 zookeeper cluster with 5 nodes
> running in virtual machines.
>      Each machine in the 43 nodes have 12 disks, one is 1TB.
>      Memory: Datanode: 2GB HMaster:8GB Regionserver: 16GB. Zookeeper: 1GB.
>      The performance is pretty good.

View raw message