hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yossi Ittach" <yoss...@gmail.com>
Subject Re: HBase region server - utilization extremely unbalanced
Date Sun, 26 Oct 2008 09:42:01 GMT
Thank you very much. I'll report my benchmark results when I have it , maybe
it will help someone

Vale et me ama
Yossi


On Thu, Oct 23, 2008 at 7:35 PM, Michael Stack <stack@duboce.net> wrote:

> Yossi Ittach wrote:
>
>> Thanks for the quick reply.
>>
>> I'm following the jvm Memory consumption (using "top") , and what bothers
>> me
>> is that it seems the percentages are just going up and up , and it makes
>> me
>> kind of worried.
>>
>>
>
> 'top' is an extremely crude tool for figuring how the JVM is doing
> memory-wise.  JVM will generally tend to grow to fill alloted space and then
> do the extra GC'ing to keep within the bound.  If you want to watch it in
> action, enable GC logging -- add "-Xloggc:/tmp/gc.log" to the hbase JVM
> options in hbase-env.sh or turn on JMX and connect to your running
> regionserver with jconsole.
>
> If you'just testing and worried about memory usage, just load till you
> OOME.  I've been able to load 30+Million rows/~400 regions into a single 1G
> RegionServer before it OOME'd.  But I've heard of others with > one family
> who have only been able to load 30 or 40 into 1G heap.
>
> St.Ack
>
>
>
>  I'm trying to load the system with 30GB of data (this is a benchmark) . I
>> estimate that my production environment will require at least 3 times that
>> size.  Is there a rule-of-thumb as to how many region servers I'll need?
>>
>>
>> Vale et me ama
>> Yossi
>>
>>
>> On Thu, Oct 23, 2008 at 2:14 PM, Jean-Daniel Cryans <jdcryans@apache.org
>> >wrote:
>>
>>
>>
>>> Yossi,
>>>
>>> The META region is usually heavily used and it's worst when you use the
>>> web
>>> UI. Just for the lolz, go on the Master's page (the main page) and hit
>>> "refresh" a couple of times; you should see that number go high up.
>>>
>>> And on how to avoid it, well the only way to split that load would be to
>>> have the META region do a split but it will require a lot of data hence a
>>> lot of user regions which I doubt you have on 2 machines.
>>>
>>> J-D
>>>
>>> On Thu, Oct 23, 2008 at 8:08 AM, Yossi Ittach <yossale@gmail.com> wrote:
>>>
>>>
>>>
>>>> Hi
>>>>
>>>> Using Hbase with 2 Region servers on similar machines , I see that one
>>>> machine is serving almost 400 requests per second , while the other one
>>>>
>>>>
>>> is
>>>
>>>
>>>> serving 0-10 . This cause extreme overload on the first machine. Any
>>>> idea
>>>> what causes it , or how it can be avoided?
>>>>
>>>> Thanks!
>>>>
>>>> Vale et me ama
>>>> Yossi
>>>>
>>>>
>>>>
>>>
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message