hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mukund murrali <mukundmurra...@gmail.com>
Subject Major compaction skipping for older regions
Date Wed, 05 Aug 2015 06:34:10 GMT

We wanted to have deleted data for a week. So we configured

TTL => 1 week.

As per our understanding, after 1 week the deleted data becomes available
for major compaction and should be purged (correct if wrong). Since we have
time series data, we don't have any write operations in those regions after
a week . But major compaction never took place for any regions and our
overall size grew drastically though we have deletes happening. After
analyzing, we found that major compaction takes place if any one of the 2
condition is satisfied.

1. If the time interval between major compaction is greater than a week
(default config).
2. if the block locality index falls below a threshold.

In our case, since we have min_versions to be 1, the first case condition
fails. Time to verify is set to Long.Max value, if min versions is not 0.

Second is block locality.  To check the block locality index we enabled
fine logs. And we found the  block locality is always 1, and we got logs
stating "Skipping major compaction......".

So, in this case is manually triggering major compaction the only choice?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message