hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bradford Stephens <bradfordsteph...@gmail.com>
Subject Re: HBase Write to Regionservers behavior
Date Fri, 12 Jun 2009 05:53:45 GMT
Oh, I misspoke. The MR job is over tab-delimited text files. I have 14
mappers and 7 reducers -- loading into an empty table. The total
amount of regions generated after the job is done and some splits
happen is 70.

On Thu, Jun 11, 2009 at 10:47 PM, stack<stack@duboce.net> wrote:
> Is your MR job over the whole table or a subset?  If whole table then its
> odd that the 7 reducers are hitting only 2 regions.  What happens if 70
> reducers?
> St.Ack
>
> On Thu, Jun 11, 2009 at 10:42 PM, Bradford Stephens <
> bradfordstephens@gmail.com> wrote:
>
>> About 70.
>>
>> On Thu, Jun 11, 2009 at 10:24 PM, stack<stack@duboce.net> wrote:
>> > Hey, how many regions?  (smile)
>> > St.Ack
>> >
>> > On Thu, Jun 11, 2009 at 10:19 PM, Bradford Stephens <
>> > bradfordstephens@gmail.com> wrote:
>> >
>> >> I meant, here 'till Midnight :) thanks!
>> >>
>> >> On Thu, Jun 11, 2009 at 10:19 PM, Bradford
>> >> Stephens<bradfordstephens@gmail.com> wrote:
>> >> > That actually make a lot of sense. Thanks, awesome people! Me and the
>> >> > dev team are here to get Katta + HBase to play together, and it's
>> >> > looking pretty nice.
>> >> >
>> >> > On Thu, Jun 11, 2009 at 9:47 PM, stack<stack@duboce.net> wrote:
>> >> >> On Thu, Jun 11, 2009 at 6:10 PM, Bradford Stephens <
>> >> >> bradfordstephens@gmail.com> wrote:
>> >> >>
>> >> >>>
>> >> >>> What I'm noticing is that it's writing to mostly one or two
regions
>> on
>> >> >>> one box at a time, even though I have 7 reducers running. Monitoring
>> >> >>> everything with dstat -v, I notice that only 2 of my servers
are
>> doing
>> >> >>> much. These boxes have very low CPU idling, and high disk output
(a
>> >> >>> few GB a minute).
>> >> >>>
>> >> >>
>> >> >>
>> >> >> How many regions in your table?
>> >> >>
>> >> >> At first, there is one.  All reducers will go against it.   When
it
>> >> splits,
>> >> >> then two regions field the 7 reducers and so on.
>> >> >>
>> >> >> You can manually split regions from the command-line.  See if
that
>> >> helps:
>> >> >>
>> >> >> hbase> split_region 'REGIONNAME'
>> >> >>
>> >> >> (IIRC -- type 'tools' in shell for help on the admin facilities).
>> >> >>
>> >> >> St.Ack
>> >> >>
>> >> >
>> >>
>> >
>>
>

Mime
View raw message