hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akmal Abbasov <akmal.abba...@icloud.com>
Subject HBase snapshots and Compaction
Date Tue, 12 May 2015 16:52:05 GMT
HI, 
I am using HBase 0.98.7.
I am using HBase snapshots to backup data. I create snapshot of tables each our.
Each create snapshot process will cause the flush of the memstore, and creation of hfiles.
When the number of hfiles will reach 3 the MINOR compaction process will start for each CF.
Ok, I was expecting that the compaction will process only small hfiles, and I won’t have
problems with moving all data to archive folder
each time after compaction process ends.
But most of the times, the minor compaction is promoted to major(more than 100 in 24 hours
without loads). 
As far as I know, the only possibility for this is that all hfiles are eligible for compaction.
But when I tested the archive folder for a CF I see the strange situation
-rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 06:04 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/36dc06f4c34242daadc343d857a35734
-rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 06:04 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/7e8b993f97b84f4594542144f15b0a1e
-rw-r--r--   3 akmal supergroup      1.1 K 2015-05-10 06:04 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/b9afc64792ba4bf99a08f34033cc46ac
-rw-r--r--   3 akmal supergroup    638.4 K 2015-05-10 06:04 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/dff846ae4fc24d418289a95322b35d46
-rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 08:50 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/228eee22c32e458e8eb7f5d031f64b58
-rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 08:50 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/529257432308466f971e41db49ecffdf
-rw-r--r--   3 akmal supergroup    638.5 K 2015-05-10 08:50 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/839d3a6fc523435d8b44f63315fd11b8
-rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 08:50 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/8c245e8661b140439e719f69a535d57f
-rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 11:37 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/23497a31d3e721fe9b63c58fbe0224d5
-rw-r--r--   3 akmal supergroup    638.7 K 2015-05-10 11:37 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/8c9af0357d164221ad46b336cd660b30
-rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 11:37 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/8eb55b43c22d434954e2e0bfda656018
-rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 11:37 /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/b8b6210d9e6d4ec2344238c6e9c17ddf

As I understood this files were copied to archive folder after compaction. 
The part I didn’t understand is, why the file with 638 K was also selected for compaction?
Any ideas?
Thank you.

Kind regards,
Akmal Abbasov



Mime
View raw message