hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Austin Heyne <ahe...@ccri.com>
Subject Compactions after bulk load
Date Tue, 17 Jul 2018 18:12:00 GMT
Hi all,

I'm trying to bulk load a large amount of data into HBase. The bulk load 
succeeds but then HBase starts running compactions. My input files are 
typically ~5-6GB and there are over 3k files. I've used the same table 
splits for the bulk ingest and the bulk load so there should be no 
reason for hbase to run any compactions. However, I'm seeing it first 
start compacting the hfiles into 25+GB files and then into 200+GB files 
but didn't let it run any longer. Additionally, I've talked with another 
coworker who's tried this process in the past and he's experience the 
same thing, eventually giving up on the feature. My attempts have been 
on HBase 1.4.2. Does anyone have information on why HBase is insisting 
on running these compactions or how I can stop them? They are 
essentially breaking the feature for us.

Thanks,

-- 
Austin L. Heyne


Mime
View raw message