hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Sechrist <ssechr...@gmail.com>
Subject Re: disable table, alter table, and merge regions
Date Wed, 01 Dec 2010 14:09:26 GMT
Hey Abhijit,

We ran into this same issue a while back, and here is what we did (and it
seemed to work ok):

1. Went onto to the HBase web ui for our biggest table, and grabbed all of
the region names (they appear in order on that page). Saved the region names
to a text file.
2. Wrote a shell script to run the hbase merge tool on every pair of regions
in that file.
3. Shut down HBase.
4. Run that shell script. It went at about 50 merges/hour on our 5 node
cluster.
5. Start HBase. When it went back up we saw that our region count was about
1500 regions, down from almost 3000.

So this would only work if you can take down HBase for a decent amount of
time.

I wonder if you could alternatively, run an Export job and an Import job of
your table. Do those preserve the regions, or could you use it to bring down
the number of regions?

-Sean

On Wed, Dec 1, 2010 at 2:24 AM, Abhijit Pol <apol@rocketfuel.com> wrote:

> We have HBase cluster which was peacefully (acceptable throughput and
> latencies) serving for about a month (we are using 0.89.20100926 version)
>
> This morning we wanted to set TTL to value smaller than default and Mr.
> Murphy struck.
>
> (A) We disabled and altered table with desired TTL value (using shell).
> Alerting one property (in this case TTL) reseted all other
> table properties to default values. For example version was reset to 3,
> compression was reset to NONE etc. [I think this is known issue with open
> Jira]
>
> (B) We wanted to go back to previous table properties. Now after multiple
> retires we were not able to disable table (even restarting cluster didn't
> helped). Most likely if clients are hitting hard (in this case ~10k qps) on
> HBase table, it takes forever to disable a table.
>
> So we stopped all clients and then were able to disable table and altered
> table properties to desired values.
>
> (C) Due to compression was reset to NONE and version was reset to 3 for
> good
> 10-12hrs, the total number of regions tripled and load (#regions/RS)
> increased from 100 to 300. After first major_compaction, compression,
> version 1, and new lower TTL became effective and we were back to original
> HDFS footprint and have bunch of small regions.
>
> What will trigger merging of these regions? the tool for merging does not
> seem to work or even if it does, it can only do two regions at-a-time.
>
> Any suggestions on how we can reduce number of regions and bring load back
> to where it was before?
>
>
> Thanks,
> --Abhi
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message