hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Balance Regions Faster
Date Wed, 21 Mar 2018 01:11:35 GMT
Please consider tuning the following parameters of stochastic load balancer
:

"hbase.master.balancer.stochastic.maxRunningTime"

default value is 30 seconds. It controls the duration of runtime for
each balanceCluster()
call.

"hbase.balancer.period"

default is 300 seconds. It controls the maximum time master runs balancer
for.

You can turn on DEBUG logging and observe the following output in master
log:

balancer.StochasticLoadBalancer: Finished computing new load balance
plan.  Computation took 1200227ms to try 2254 different iterations.
Found a solution that moves 550 regions; Going from a computed cost of
77.52829271038965 to a new cost of 74.32764924425548

If you have a dev cluster, you can try different combinations of the above
two parameters and get best performance by checking the above log.

Cheers

On Tue, Mar 20, 2018 at 5:18 PM, Saad Mufti <saad.mufti@gmail.com> wrote:

> Hi,
>
> We are using the stochastic load balancer, and have tuned it to do a
> maximum of 1% of regions in any calculation. But it is way too conservative
> after that, it moves one region at a time. Is there a way to tell it to go
> faster with whatever number of regions it decided to do? I have been
> looking at the settings in the code but so far a little confused about
> which exact setting will achieve this. Is it one of the steps settings?
>
> Also our cluster was in bad shape for a while as a bunch of region servers
> aborted for different reasons. When we stopped everything and brought it
> back up and enabled the tables, most of the region servers were assigned no
> regions. Is it for locality reasons that HBase is trying to assign regions
> where they were assigned before? Is there a way to tell HBase to ignore
> that on startup?
>
> Thanks.
>
> ----
> Saad
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message