hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saad Mufti <saad.mu...@gmail.com>
Subject Re: Balance Regions Faster
Date Wed, 21 Mar 2018 03:01:03 GMT
Thanks, will take a look. We're using HBase 1.4.0 on AWS EMR.

Cheers.

----
Saad


On Tue, Mar 20, 2018 at 9:32 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Saad:
> You didn't mention the version of hbase you are using.
> Please check the version to see if the following were included:
>
> HBASE-18164 Fast locality computation in balancer
> HBASE-16570 Compute region locality in parallel at startup
> HBASE-15515 Improve LocalityBasedCandidateGenerator in Balancer
>
> Cheers
>
> On Tue, Mar 20, 2018 at 6:11 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Please consider tuning the following parameters of stochastic load
> > balancer :
> >
> > "hbase.master.balancer.stochastic.maxRunningTime"
> >
> > default value is 30 seconds. It controls the duration of runtime for
> each balanceCluster()
> > call.
> >
> > "hbase.balancer.period"
> >
> > default is 300 seconds. It controls the maximum time master runs balancer
> > for.
> >
> > You can turn on DEBUG logging and observe the following output in master
> > log:
> >
> > balancer.StochasticLoadBalancer: Finished computing new load balance
> plan.  Computation took 1200227ms to try 2254 different iterations.  Found
> a solution that moves 550 regions; Going from a computed cost of
> 77.52829271038965 to a new cost of 74.32764924425548
> >
> > If you have a dev cluster, you can try different combinations of the
> above
> > two parameters and get best performance by checking the above log.
> >
> > Cheers
> >
> > On Tue, Mar 20, 2018 at 5:18 PM, Saad Mufti <saad.mufti@gmail.com>
> wrote:
> >
> >> Hi,
> >>
> >> We are using the stochastic load balancer, and have tuned it to do a
> >> maximum of 1% of regions in any calculation. But it is way too
> >> conservative
> >> after that, it moves one region at a time. Is there a way to tell it to
> go
> >> faster with whatever number of regions it decided to do? I have been
> >> looking at the settings in the code but so far a little confused about
> >> which exact setting will achieve this. Is it one of the steps settings?
> >>
> >> Also our cluster was in bad shape for a while as a bunch of region
> servers
> >> aborted for different reasons. When we stopped everything and brought it
> >> back up and enabled the tables, most of the region servers were assigned
> >> no
> >> regions. Is it for locality reasons that HBase is trying to assign
> regions
> >> where they were assigned before? Is there a way to tell HBase to ignore
> >> that on startup?
> >>
> >> Thanks.
> >>
> >> ----
> >> Saad
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message