hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Brown <tombrow...@gmail.com>
Subject Too many "Compaction Complete" messages
Date Wed, 04 Sep 2013 02:13:42 GMT
I have a very small cluster that holds a lot of data. In fact, it's almost
at it's capacity from data size and ops/second perspectives. There are two
region servers, each with about 180 regions (too many, I know) and 16GB of
RAM, 8 of which is dedicated to HBase. I am running HBase 0.94.10.

Recently, the web UI went unresponsive. I couldn't tell what was really
happening from the logs so I attempted to restart the region servers. They
did not respond to a normal shutdown request within 10 minutes so I killed
them with SIGKILL. I also increased the region split size from 2GB to 4GB,
to avoid increasing the region count.

There were no reported problems or errors when I started back up, but ever
since then I have had hundreds of "Compacting d in {sometable}......
Compaction Complete" messages filling up my region server task status. It
has been doing this with no appreciable change in pace for 5 days now.

When I look at the regionserver web UI, I see that many regions have a
numeric value for compactionProgressPct, but none is higher than 10.0.

There isn't enough free space to rewrite/compact any entire table, but
there should be enough free space to rewrite/compact any single region
(even multiple regions at once).

Any ideas why my task status logs might be filling up like that? How can I
verify that it's really compacting like it says? Will it ever finish?

Thanks in advance,
Tom Brown

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message