hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Latham <lat...@davelink.net>
Subject Re: why Hbase only split regions in one RegionServer
Date Wed, 16 Mar 2016 15:20:55 GMT
You're definitely better off today if you know what your data looks like
and are able to set up your table ahead of time accordingly.  But I think
this is an area where HBase can and should do better.  We should try to
lower the bar and make it simpler for people to get started.

HBase would likely eventually balance things out, but if it's unable to do
that while the single server is under load, then it's not serving this case
for newcomers very well.

On Wed, Mar 16, 2016 at 8:07 AM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> It's usually better to do a bit more work upstream to know about you table
> schema and keys instead of trying to fight that later.
>
> HBase will handle the case. As soon as the compactions will be done. It's
> just that it's not a recommanded way to proceed. Better to prevent instead
> of trying to fix it.
>
> JMS
>
> 2016-03-16 10:54 GMT-04:00 Dave Latham <latham@davelink.net>:
>
> > What if someone doesn't know the distribution of their row keys?
> > HBase should be able to handle this case.
> >
> > On Wed, Mar 16, 2016 at 7:18 AM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org> wrote:
> >
> > >  Balancer is not moving regions that are compacting, right? He is just
> > > pusing to much load on a non splitted table that will keep splitting
> and
> > > compacting like crazy until balancer get a chance to get in action.
> > >
> > > Pre-split / Balance. Problem solved.
> > >
> > > Jack, when the ingestion of data is done, is the table balancing? Any
> > other
> > > table on the cluster? How many regions per region server?
> > >
> > > JMS
> > >
> > > 2016-03-16 10:16 GMT-04:00 Ted Yu <yuzhihong@gmail.com>:
> > >
> > > > In Jack's case, even if the table was pre-split, after loading some
> > data,
> > > > if balancer didn't run, the regions would still be out of balance.
> > > >
> > > > We should help Jack find out the cause for imbalance of regions.
> > > >
> > > > On Wed, Mar 16, 2016 at 4:17 AM, Jean-Marc Spaggiari <
> > > > jean-marc@spaggiari.org> wrote:
> > > >
> > > > > +1 with what Heng said. I think we should just deprecate the
> ability
> > to
> > > > not
> > > > > pre-split a table ;) It's always good to pre-split it based on your
> > key
> > > > > design...
> > > > >
> > > > > 2016-03-16 0:17 GMT-04:00 Heng Chen <heng.chen.1986@gmail.com>:
> > > > >
> > > > > > bq. the table I created by default having only one region
> > > > > >
> > > > > > Why not pre-split table into more regions when create it?
> > > > > >
> > > > > > 2016-03-16 11:38 GMT+08:00 Ted Yu <yuzhihong@gmail.com>:
> > > > > >
> > > > > > > When one region is split into two, both daughter regions
are
> > opened
> > > > on
> > > > > > the
> > > > > > > same server where parent region was opened.
> > > > > > >
> > > > > > > Can you provide a bit more information:
> > > > > > >
> > > > > > > release of hbase
> > > > > > > whether balancer was turned on - you can inspect master
log to
> > see
> > > if
> > > > > > > balancer was on
> > > > > > >
> > > > > > > Consider pastebinning portion of master log.
> > > > > > >
> > > > > > > Thanks
> > > > > > >
> > > > > > > On Tue, Mar 15, 2016 at 4:43 PM, jackwang <
> > wangjiajie917@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > I was writing 300GiB data to my Hbase table user_info,
the
> > table
> > > I
> > > > > > > created
> > > > > > > > by
> > > > > > > > default having only one region. When the writing was
going I
> > saw
> > > > one
> > > > > > > region
> > > > > > > > became two regions and more late on it became 8 regions.
But
> my
> > > > > > confusion
> > > > > > > > is
> > > > > > > > that the 8 regions were kept in the same RegionServer.
> > > > > > > >
> > > > > > > > Why Hbase didn't split the regions to different RegionServer.
> > > btw,
> > > > I
> > > > > > had
> > > > > > > 10
> > > > > > > > physical RegionsServers in my Hbase cluster, and the
region
> > size
> > > I
> > > > > set
> > > > > > is
> > > > > > > > 20GiB, Thanks!
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > View this message in context:
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-hbase.679495.n3.nabble.com/why-Hbase-only-split-regions-in-one-RegionServer-tp4078497.html
> > > > > > > > Sent from the HBase User mailing list archive at Nabble.com.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message