hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David chen" <c77...@163.com>
Subject Re: Why can the capacity of a table with TTL grow continuously?
Date Wed, 18 Mar 2015 07:08:46 GMT
Region size increased from 4G to 10G, and merged the adjacent region, i.e. the region number
reduced half approximately. Then the table capacity reduced from 969.4G to 945.2G, the regionServer
information are as follows:
ServerName                        Num. StoresNum. StorefilesStorefile Size UncompressedStorefile
SizeIndex SizeBloom Size
Before processing
rs1,60020,1426240567402131                254        556333m                155989mb499463k552615k
rs2,60020,1426240567533125                234        543717m                151136mb498532k542311k
rs3,60020,1426240567677131                255        555199m                155714mb504966k630839k
rs4,60020,1426240567632123                235        540522m                154388mb499349k660950k
rs5,60020,1426240567340128                243        537569m                151889mb488991k612977k
rs6,60020,1426240565314126                238        567384m                158269mb521661k576256k
rs7,60020,1426240565519118                242        569310m                155403mb528435k643266k


After processing
rs1,60020,142624056740262                53        446499m                125661mb408080k462457k
rs2,60020,142624056753368                53        516360m                144451mb470769k499128k
rs3,60020,142624056767771                66        534077m                150870mb481473k587248k
rs4,60020,142624056763275                68        587874m                166252mb546647k703632k
rs5,60020,142624056734071                59        527609m                149605mb478360k587054k
rs6,60020,142624056531463                48        501960m                140437mb466324k512596k
rs7,60020,142624056551976                63        662230m                180782mb608837k706609k
 





At 2015-03-12 10:34:10, "Alex Baranau" <alex.baranov.v@gmail.com> wrote:
>I'd try that. Please come back with results. Also, if possible it will be
>useful (at least for the important jira mentioned by Nick) if you can share
>the stats on the regions (size, store files #) before and after the
>procedure. Note (as Lars said): "careful, this ... can put some load on the
>net/disks".
>
>One more note: only increasing hbase.hregion.max.filesize may not help to
>completely avoid the same situation in future. It's hard to tell, but if
>you have what I'm thinking, I'd say data distribution pattern has greater
>affect. Though it will be mitigated to some extend by upping the region
>size.
>
>Alex Baranau
>--
>http://cdap.io - open source framework to build and run data applications on
>Hadoop & HBase
>
>On Wed, Mar 11, 2015 at 7:00 PM, David chen <c77_cn@163.com> wrote:
>
>> hbase.store.delete.expired.storefile is true in file
>> hbase-0.98.5/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compaction/CompactionConfigureation.java,
>> the reference code is 83th line as follows:
>> shouldDeleteExpired =
>> conf.getBoolean("hbase.store.delete.expired.storefile", true);
>>
>>
>> The region number have grown from 16 to 263 over the past seven months,
>> maybe the hbase.hregion.max.filesize value(4G) is a bit small. It looks
>> likely that the solution is to adjust hbase.hregion.max.filesize bigger and
>> merge the adjacent regions.
>> Any other ideas to suggest?
>>
>>
>>
>>
>>
>>
>>
>>
>>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message