hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Balance Regions Faster
Date Wed, 21 Mar 2018 01:32:28 GMT
Saad:
You didn't mention the version of hbase you are using.
Please check the version to see if the following were included:

HBASE-18164 Fast locality computation in balancer
HBASE-16570 Compute region locality in parallel at startup
HBASE-15515 Improve LocalityBasedCandidateGenerator in Balancer

Cheers

On Tue, Mar 20, 2018 at 6:11 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Please consider tuning the following parameters of stochastic load
> balancer :
>
> "hbase.master.balancer.stochastic.maxRunningTime"
>
> default value is 30 seconds. It controls the duration of runtime for each balanceCluster()
> call.
>
> "hbase.balancer.period"
>
> default is 300 seconds. It controls the maximum time master runs balancer
> for.
>
> You can turn on DEBUG logging and observe the following output in master
> log:
>
> balancer.StochasticLoadBalancer: Finished computing new load balance plan.  Computation
took 1200227ms to try 2254 different iterations.  Found a solution that moves 550 regions;
Going from a computed cost of 77.52829271038965 to a new cost of 74.32764924425548
>
> If you have a dev cluster, you can try different combinations of the above
> two parameters and get best performance by checking the above log.
>
> Cheers
>
> On Tue, Mar 20, 2018 at 5:18 PM, Saad Mufti <saad.mufti@gmail.com> wrote:
>
>> Hi,
>>
>> We are using the stochastic load balancer, and have tuned it to do a
>> maximum of 1% of regions in any calculation. But it is way too
>> conservative
>> after that, it moves one region at a time. Is there a way to tell it to go
>> faster with whatever number of regions it decided to do? I have been
>> looking at the settings in the code but so far a little confused about
>> which exact setting will achieve this. Is it one of the steps settings?
>>
>> Also our cluster was in bad shape for a while as a bunch of region servers
>> aborted for different reasons. When we stopped everything and brought it
>> back up and enabled the tables, most of the region servers were assigned
>> no
>> regions. Is it for locality reasons that HBase is trying to assign regions
>> where they were assigned before? Is there a way to tell HBase to ignore
>> that on startup?
>>
>> Thanks.
>>
>> ----
>> Saad
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message