hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: how many rows per GB memory
Date Mon, 27 Oct 2008 19:51:46 GMT
Billy Pearson wrote:
> Say I have 8 regions each with 4 families how much memory should it 
> take to load the index files and do read request on the regions 
> assuming no write will happen.
>
If no writes, then memcaches will have no content.  So, only memory load 
should be the indexes on the store files.

> Currently I have compression turned on for the family and they hold 
> around 900M rows total but use 1.3GB of
> memory when the region server is started and done loading regions for 
> the table.
> Is there a way to lower that memory usage?
>
> If I add up all the index files that are with the data mapfiles they 
> total only 173MB
> So do we hold only index data for reads if so why is my region server 
> using 1.3GB vs something like 173MB on start up? 

I'm not sure why its so much.  YOu have your heap size set to 2G?  Maybe 
the churn around startup had the heap grow out to 1.3G but now its 
settled, it might run in less?  You could try setting down your heapsize 
if you're > 1G default.  Also, enable gc logging.   That'll give you a 
better clue as to how much memory is actually being used.  Add something 
like this to your hbase-env.sh:

export HBASE_OPTS="-server -Xloggc:/tmp/gc.log"

St.Ack

Mime
View raw message