hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: Regionserver fails to serve region
Date Tue, 04 Nov 2008 19:33:51 GMT
Slava Gorelik wrote:
> Hi.I happened yesterday after 28 hours of running i started the hadoop
> balancer after 2 hours working (with some exception- ca't move block) my
> hbase started to throws exception that can't create new block. It happened
> on on couple of region servers and then master is failed to connect to then
> and eventually it crashed. Tomorrow I can cut last 2-3 hours from logs (they
> are huge) and send you.

Sorry about that.  I presumed it safe given that balancer has been 
around a few releases and we're running it here continuously w/o issue.  
Did estart fix things or are there now missing blocks?
> BTW, some i sent email to list couple of days ago about blockCache parameter
> on column family descriptor, what is it and how it affect on performance ?
Sorry, missed it.

The blockcache is client-side caching of pieces of store files.  You can 
set the size of the blocks to cache client-side.  It uses java Soft 
References.  Blocks are evicted on roughly an LRU basis when memory is 
low.  It was added a good while ago by Tom White.

By default it has been off but as of HBASE-953 commit of about a week or 
so ago, after some playing and tuning, the default has been flipped and 
now blockcache is on by default.  Block caching along with other 
performance improvements including rpc fixes and J-D's scanner 
pre-fetching and batch writing, will make the 0.19.0 release run faster 
than its predecessors in many regards.

Some rough benchmarking running our performance test -- keep in mind, 
this is not-very-real-world just a single client going against a single 
regionserver (see wiki for more) -- shows writes running at ~3X speed, 
scans at ~7X, sequential reads at ~2X and random reads anywhere from 
slower to 2 to 3 times faster dependent on how well the block cache is 
helping (or hindering).  If the regionserver has more memory, random 
reads run faster.  If not enough, regionserver is just spinning filling 
cache and random read times plummet.  I'll put up some numbers when we 
come closer to the 0.19.0 release.

If you do enable block cache, be sure to update your hbase-default.xml.  
The old block size tends to provoke OOMEs.

View raw message