hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ding,Dongchao" <dingdongc...@baidu.com>
Subject Re: Leveraging As Much Memory As Possible
Date Thu, 31 Mar 2016 06:24:24 GMT
For the  first question

http://${rs-host}:${rs-info-port}/rs-status#bc_l2, There is detail info
about L2 Cache(bucket cache)here.




在 16/3/31 14:07, "Amit Shah" <amits.84@gmail.com> 写入:

>Thanks Ted and Anoop for the replies. A couple of follow up questions
>
>1. I could see the bucket cache and on heap memory stats reflected on
>hbase
>region server UI <http://i.imgur.com/4uwaWuC.png>, what are ways to
>monitor
>and see if its getting utilized?
>2. We are using phoenix as a sql layer above hbase. Are there any phoenix
>configurations that I could benefit in leveraging memory?
>
>Thanks,
>Amit.
>
>On Thu, Mar 31, 2016 at 10:44 AM, Anoop John <anoop.hbase@gmail.com>
>wrote:
>
>> Ya having HBase side cache will be a better choice rather than HDFS
>> cache IMO.   Yes u r correct...  You might not want to give a very
>> large size for the heap. You can make use of the off heap BucketCache.
>>
>> -Anoop-
>>
>> On Thu, Mar 31, 2016 at 4:35 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>> > For #1, please see the top two blogs @ https://blogs.apache.org/hbase/
>> >
>> > FYI
>> >
>> > On Wed, Mar 30, 2016 at 7:59 AM, Amit Shah <amits.84@gmail.com> wrote:
>> >
>> >> Hi,
>> >>
>> >> I am trying to configure my hbase (version 1.0) phoenix (version -
>>4.6)
>> >> cluster to utilize as much memory as possible on the server
>>hardware. We
>> >> have an OLAP workload that allows users to perform interactive
>>analysis
>> >> over huge sets of data. While reading about hbase configuration I
>>came
>> >> across two configs
>> >>
>> >> 1. Hbase bucket cache
>> >> <
>> >>
>> 
>>http://blog.asquareb.com/blog/2014/11/24/how-to-leverage-large-physical-m
>>emory-to-improve-hbase-read-performance
>> >> >
>> >> (off heap) which looks like a good option to bypass garbage
>>collection.
>> >> 2. Hadoop pinned hdfs blocks
>> >> <
>> http://blog.cloudera.com/blog/2014/08/new-in-cdh-5-1-hdfs-read-caching/>
>> >> (max locked memory) concept that loads the hdfs blocks in memory, but
>> given
>> >> that hbase is configured with short circuit reads I assume this
>>config
>> may
>> >> not be of much help. Instead it would be right to increase hbase
>>region
>> >> server heap memory. Is my understanding right?
>> >>
>> >> We use HBase with Phoenix.
>> >> Kindly let me know your thoughts or suggestions on any more options
>> that I
>> >> should explore
>> >>
>> >> Thanks,
>> >> Amit.
>> >>
>>

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message