hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Slava Gorelik" <slava.gore...@gmail.com>
Subject Re: Regionserver fails to serve region
Date Tue, 04 Nov 2008 18:20:20 GMT
Hi.I happened yesterday after 28 hours of running i started the hadoop
balancer after 2 hours working (with some exception- ca't move block) my
hbase started to throws exception that can't create new block. It happened
on on couple of region servers and then master is failed to connect to then
and eventually it crashed. Tomorrow I can cut last 2-3 hours from logs (they
are huge) and send you.

Thank You for Your assistance.

BTW, some i sent email to list couple of days ago about blockCache parameter
on column family descriptor, what is it and how it affect on performance ?

Best Regards.



On Tue, Nov 4, 2008 at 7:27 PM, stack <stack@duboce.net> wrote:

> Did it?  It shouldn't.  If it does, then its a bug we need to figure.
>  Whats its log say?  Otherwise you could do the rebalance manually but your
> results will be spotter (Up replication, let it run a while, then
> selectively shutdown nodes checking filesystem as you go to make sure at
> least one replica is still being served and then restore original
> replication level, etc.).
> St.Ack
>
> Slava Gorelik wrote:
>
>> Can i run balancer on hadoop during massive load on Hbase ?Last time i did
>> it i killed my data :-)
>>
>> On Tue, Nov 4, 2008 at 10:45 AM, Michael Stack <stack@duboce.net> wrote:
>>
>>
>>
>>> Slava Gorelik wrote:
>>>
>>>
>>>
>>>> Hi Michael.After reformatting HDFS, Hbase started to work as a Swiss
>>>> Clock.
>>>> Worked with 8 clients about 30 hours intensive load.
>>>>
>>>>
>>>>
>>>>
>>> Thanks for reporting back to the list.
>>>
>>>
>>>
>>>> Just small question, after about 28 hours (when i came back to work) i
>>>> found
>>>> that one of 7 datanodes in Hadoop is about 98% usage and all other about
>>>> 30%, is it normal ?
>>>>
>>>>
>>>>
>>>>
>>> I haven't kept a close eye on HDFS usage during hbase upload.  I do know
>>> that out-of-balance would seem to be a common condition and that its been
>>> reported on the list that hbase runs faster on a balanced HDFS.  You
>>> might
>>> try out the hadoop balancer.  There's a little note on it here in the
>>> hbase
>>> FAQ: http://wiki.apache.org/hadoop/Hbase/FAQ#7.
>>>
>>> St.Ack
>>>
>>>
>>>
>>
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message