hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "donhoff_h" <165612...@qq.com>
Subject 回复: BucketCache Configuration Problem
Date Mon, 02 Mar 2015 14:19:42 GMT
Hi, Stack

Thanks for your reply and also thanks very much for your reply to my previous mail "Questions
about BucketCache".

The hbase.bucketcache.bucket.sizes property takes effect. I did not notice that in your first
mail you have told me the name should be hbase.bucketcache.bucket.sizes, instead of hbase.bucketcache.sizes.
I noticed this point until your last mail. It's my fault. Thanks for your patient.

As to the relationship between HBASE_OFFHEAPSIZE and -XX:MaxDirectMemorySize, I followed your
advice to find in the bin/hbase the statements that contain HBASE_OFFHEAPSIZE, but I found
that there isn't any statement which contains HBASE_OFFHEAPSIZE. I also tried "bin/hbase-daemon.sh"
and "bin/hbase-config.sh", they don't contain HBASE_OFFHEAPSIZE either. So I still don't know
their relationship. My HBase version is 0.98.10. Is HBASE_OFFHEAPSIZE not used in this version

As to "the pure secondary cache" or "bypass CBC", I mean use BucketCache as a strict L2 cache
to the L1 LruBlockCache, ie the Raw L1+L2. The point comes from the reference guide of Apache
HBase, which says : "It is possible to deploy an L1+L2 setup where we bypass the CombinedBlockCache
policy and have BucketCache working as a strict L2 cache to the L1 LruBlockCache. For such
a setup, set CacheConfig.BUCKET_CACHE_COMBINED_KEY to false. In this mode, on eviction from
L1, blocks go to L2. When a block is cached, it is cached first in L1. When we go to look
for a cached block, we look first in L1 and if none found, then search L2. Let us call this
deploy format, Raw L1+L2."
I want to configure this kind of cache not because the CBC poliy is not good, but because
I am a tech-leader in a bank. I need to compare these two kinds of cache to make a decision
for our differenct kinds of apps. The reference guide said I can configure in "CacheConfig.BUCKET_CACHE_COMBINED_KEY‍",
but is there anyway I can configure in hbase-site.xml? 

Many Thanks!‍

------------------ 原始邮件 ------------------
发件人: "Stack";<stack@duboce.net>;
发送时间: 2015年3月2日(星期一) 下午2:30
收件人: "Hbase-User"<user@hbase.apache.org>; 

主题: Re: BucketCache Configuration Problem

On Sun, Mar 1, 2015 at 12:57 AM, donhoff_h <165612158@qq.com> wrote:

> Hi, experts
> I am using HBase0.98.10 and have 3 problems about BucketCache
> configuration.
> First:
> I  read the reference guide of Apache HBase to learn how to config
> BucketCache. I find that when using offheap BucketCache, the reference
> guide says that I should config the HBASE_OFFHEAPSIZE , it also says  that
> I should config -XX:MaxDirectMemorySize. Since these two parameters  are
> both related to the DirectMemory, I am confused which one should I
> configure?
See bin/hbase how HBASE_OFFHEAPSIZE gets interpolated as value of the
-XX:MaxDirectMemorySize passed to java (so set HBASE_OFFHEAPSIZE) (will fix
the doc so more clear)

> Second:
> I want to know how to configure the  BucketCache as a pure secondary
> cache, which I mean to bypass the  CombinedBlockCache policy. I tried to
> configure as following , but when I  go to the regionserver's webUI, I
> found it says "No L2 deployed"
> hbase.bucketcache.ioengine=offheap
> hbase.bucketcache.size=200
> hbase.bucketcache.combinedcache.enabled=false
What do you mean by pure secondary cache? Which block types do you want in
the bucketcache?

Why bypass CBC? We've been trying to simplify bucketcache deploy. Part of
this streamlining has been removing the myriad options because they tend to
confuse and give user a few simple choices. Do the options not work for you?

> Third:
> I  made the following configuration to set the Bucket Sizes. But from
> regionserver's WebUI, I found that (4+1)K and (8+1)K sizes are used,
> (64+1)K sizes are not used. What's wrong with my configuration?
> hbase.bucketcache.ioengine=offheap
> hbase.bucketcache.size=200
> hbase.bucketcache.combinedcache.enabled=true
> hbase.bucketcache.sizes=65536  hfile.block.cache.sizes=65536 I configured
> these two both for I don't  know which one is in use now.
As per previous mail (and HBASE-13125), hfile.block.cache.sizes has no
effect in 0.98.x.  Also per our previous mail "Questions about
BucketCache", isn't it hbase.bucketcache.bucket.sizes that you want?

You have tried the defaults and they do not fit your access patten?


> Many Thanks!‍
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message