hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Monitor off heap Bucket Cache
Date Mon, 01 Jun 2015 15:10:59 GMT
On Mon, Jun 1, 2015 at 2:24 AM, Dejan Menges <dejan.menges@gmail.com> wrote:

> Hi,
> What's the best way to monitor / know how's bucket cache being used, how
> much stuff is cached there, etc?
See the UI on a regionserver. Look down the page to the 'Block Cache'
section. It has detail on both onheap and LRU offheap. See also the
documentation on offheap cache:
http://hbase.apache.org/book.html#offheap.blockcache Look also at metrics
where we report block cache stats as well as offheap used by the JVM.

> Our RegionServer can use 32G of heap size, so we exported HBASE_OFFHEAPSIZE
> to 24G in hbase-env.sh, set hfile.block.cache.size to 0.05, and set couple
> of block sizes that we know we are using knowing our usage patterns. And
> this is where strange part starts - in web UI we see now, with turning this
> off,

Turning off what?

> with those values, that total BlockCache available is 1G - before it
> was 10G. What we basically tried to achieve was to double it to 20G.
Are you sure this not the onheap portion of BlockCache?

> Documentation we were referring to was
> http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/admin_hbase_blockcache_configure.html#concept_cp3_fhy_dr_unique_1__section_m3r_2cz_dr_unique_1
> as HBase book is not going into too much details how to properly configure
> this and get what you want.
Please let us know what is missing from here:
http://hbase.apache.org/book.html#offheap.blockcache  We'd like to fix it.

> Btw. if we put hfile.block.cache.size to 0.2 we see in web UI that total
> available cache is 24G, but then after some time we had region server
> crashing. Host server have enough RAM, considering only data node and
> region server running there (128G of RAM in total) so we thought we could
> increase caching with turning on this functionality.
Make sure your hbase has HBASE-11678

> Do you maybe see what exactly we are doing wrong? How to exactly increase
> offheap caching - and is it possible to monitor it anyhow, as in metrics we
> don't see anything associating to it?
Please provide more detail. Your configurations and what you've changed in


> Thanks,
> Dejan

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message