hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luke Forehand <luke.foreh...@networkedinsights.com>
Subject Re: Hanging regionservers
Date Fri, 16 Jul 2010 02:18:06 GMT
I'm a noob.  I've pasted the region server log here:

2010-07-15 16:22:19,716 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/813182176, seqid=25559248072
2010-07-15 16:22:20,485 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/813182176/tf/8345194678673310700; store size is 96.5m
2010-07-15 16:22:20,486 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:20,486 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1263056940000:com.spacefem.www:013postfeed11020100109161214.gz:800,1279168489011 in 0sec
2010-07-15 16:22:20,486 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1255890480000:com.pbnation.www:001postfeed10320091019002445.gz:715,1279153889143
2010-07-15 16:22:20,488 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 145.4m; Skipped 4 files, size: 152270605
2010-07-15 16:22:20,490 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 79.8m; Skipped 4 files, size: 83563231
2010-07-15 16:22:20,490 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:20,493 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1255890480000:com.pbnation.www:001postfeed10320091019002445.gz:715,1279153889143 in 0sec
2010-07-15 16:22:20,493 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1256342400000:com.babycenter:004postfeed10520091026010247.gz:534,1279062922023
2010-07-15 16:22:20,495 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 197.8m; Skipped 2 file(s), size: 196832205
2010-07-15 16:22:20,496 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/771109626, seqid=25559426687
2010-07-15 16:22:21,916 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/771109626/core/4299106562423672142; store size is 197.7m
2010-07-15 16:22:21,917 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 115.7m; Skipped 2 file(s), size: 118365216
2010-07-15 16:22:21,917 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/771109626, seqid=25559426687
2010-07-15 16:22:23,031 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/771109626/tf/2388945964748880840; store size is 115.7m
2010-07-15 16:22:23,032 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:23,032 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1256342400000:com.babycenter:004postfeed10520091026010247.gz:534,1279062922023 in 2sec
2010-07-15 16:22:23,032 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1256472240000:com.cyclonefanatic.www:005postfeed10220091025181203.gz:278,1279169987222
2010-07-15 16:22:23,034 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 137.6m; Skipped 4 file(s), size: 143014117
2010-07-15 16:22:23,034 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/671106177, seqid=25573465576
2010-07-15 16:22:23,227 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/671106177/core/5230322465523887917; store size is 137.6m
2010-07-15 16:22:23,228 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 79.7m; Skipped 3 file(s), size: 82980786
2010-07-15 16:22:23,228 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/671106177, seqid=25573465576
2010-07-15 16:22:23,367 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/671106177/tf/1589202054953533902; store size is 79.7m
2010-07-15 16:22:23,368 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:23,368 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1256472240000:com.cyclonefanatic.www:005postfeed10220091025181203.gz:278,1279169987222 in 0sec
2010-07-15 16:22:23,368 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1256679300000:com.treocentral.www:001postfeed10520091028042305.gz:189,1279153648413
2010-07-15 16:22:23,370 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 165.5m; Skipped 2 file(s), size: 164037939
2010-07-15 16:22:23,370 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1052691158, seqid=25559562377
2010-07-15 16:22:24,435 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1052691158/core/6636626033924262001; store size is 165.4m
2010-07-15 16:22:24,436 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 94.4m; Skipped 2 file(s), size: 94075381
2010-07-15 16:22:24,437 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1052691158, seqid=25559562377
2010-07-15 16:22:25,348 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1052691158/tf/2464774681091383835; store size is 94.3m
2010-07-15 16:22:25,348 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:25,349 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1256679300000:com.treocentral.www:001postfeed10520091028042305.gz:189,1279153648413 in 1sec
2010-07-15 16:22:25,349 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1259841900000:com.nasioc.www:001postfeed10720091205021004.gz:331,1279225014681
2010-07-15 16:22:25,351 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 142.4m; Skipped 3 files, size: 147985964
2010-07-15 16:22:25,352 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 84.9m; Skipped 3 files, size: 88270348
2010-07-15 16:22:25,352 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:25,352 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1259841900000:com.nasioc.www:001postfeed10720091205021004.gz:331,1279225014681 in 0sec
2010-07-15 16:22:25,352 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1262220540000:com.animesuki.www:013postfeed11120100107181653.gz:797,1279169281498
2010-07-15 16:22:25,354 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 157.3m; Skipped 3 files, size: 164507378
2010-07-15 16:22:25,355 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 82.2m; Skipped 3 files, size: 85970088
2010-07-15 16:22:25,355 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:25,356 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1262220540000:com.animesuki.www:013postfeed11120100107181653.gz:797,1279169281498 in 0sec
2010-07-15 16:22:25,356 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1256486640000:ca.digitalhome.www:002postfeed10520091026070416.gz:511,1279169062627
2010-07-15 16:22:25,358 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 143.7m; Skipped 2 files, size: 150271654
2010-07-15 16:22:25,359 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 84.8m; Skipped 2 files, size: 88726688
2010-07-15 16:22:25,359 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:25,359 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1256486640000:ca.digitalhome.www:002postfeed10520091026070416.gz:511,1279169062627 in 0sec
2010-07-15 16:22:25,360 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258429740000:com.sigforum.www:006postfeed10320091118040804.gz:32,1279152662332
2010-07-15 16:22:25,362 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 245.7m; Skipped 2 file(s), size: 244807399
2010-07-15 16:22:25,362 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/766713658, seqid=25560083186
2010-07-15 16:22:26,711 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/766713658/core/7951654095037300844; store size is 245.7m
2010-07-15 16:22:26,712 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 147.9m; Skipped 3 files, size: 154650470
2010-07-15 16:22:26,712 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:26,713 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258429740000:com.sigforum.www:006postfeed10320091118040804.gz:32,1279152662332 in 1sec
2010-07-15 16:22:26,713 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1263186780000:com.classiczcars.www:003postfeed11120100112021611.gz:553,1279225689006
2010-07-15 16:22:26,714 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 141.6m; Skipped 2 files, size: 147424769
2010-07-15 16:22:26,715 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 87.8m; Skipped 2 files, size: 91429490
2010-07-15 16:22:26,715 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:26,716 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1263186780000:com.classiczcars.www:003postfeed11120100112021611.gz:553,1279225689006 in 0sec
2010-07-15 16:22:26,716 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1255517160000:com.ni.www:004postfeed11120091019172709.gz:647,1278984683928
2010-07-15 16:22:26,718 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 206.5m; Skipped 3 files, size: 216112612
2010-07-15 16:22:26,719 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 122.9m; Skipped 3 files, size: 128597614
2010-07-15 16:22:26,719 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:26,720 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1255517160000:com.ni.www:004postfeed11120091019172709.gz:647,1278984683928 in 0sec
2010-07-15 16:22:26,720 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258224300000:com.fool.www:001postfeed10720091114210442.gz:283,1279152889307
2010-07-15 16:22:26,722 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 222.3m; Skipped 1 file(s), size: 192619183
2010-07-15 16:22:26,722 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/850279304, seqid=25560287060
2010-07-15 16:22:31,155 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/850279304/core/8565501918226183399; store size is 222.1m
2010-07-15 16:22:31,156 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 122.6m; Skipped 1 file(s), size: 108092017
2010-07-15 16:22:31,156 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/850279304, seqid=25560287060
2010-07-15 16:22:34,550 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1510699, Hit=0, Miss=1510699, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN
2010-07-15 16:22:34,780 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/850279304/tf/1200271946316859038; store size is 122.6m
2010-07-15 16:22:34,780 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:34,782 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258224300000:com.fool.www:001postfeed10720091114210442.gz:283,1279152889307 in 8sec
2010-07-15 16:22:34,782 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257159300000:com.apple.www:004postfeed11020091102201418.gz:639,1279082071003
2010-07-15 16:22:34,784 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 240.9m; Skipped 4 file(s), size: 250848585
2010-07-15 16:22:34,784 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/1774135628, seqid=25571419840
2010-07-15 16:22:35,009 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1774135628/core/7916644627160380675; store size is 240.9m
2010-07-15 16:22:35,010 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 144.6m; Skipped 4 file(s), size: 150799228
2010-07-15 16:22:35,010 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/1774135628, seqid=25571419840
2010-07-15 16:22:35,177 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1774135628/tf/790350627533118818; store size is 144.6m
2010-07-15 16:22:35,177 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:35,178 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257159300000:com.apple.www:004postfeed11020091102201418.gz:639,1279082071003 in 0sec
2010-07-15 16:22:35,178 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1254612000000:org.audiokarma.www:001postfeed10220091004111818.gz:180,1278993897754
2010-07-15 16:22:35,179 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 104.1m; Skipped 2 file(s), size: 109098607
2010-07-15 16:22:35,180 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/2118757872, seqid=25560807788
2010-07-15 16:22:35,252 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2118757872/core/526802940730195272; store size is 104.1m
2010-07-15 16:22:35,253 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 49.7m; Skipped 3 files, size: 52105359
2010-07-15 16:22:35,253 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:35,254 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1254612000000:org.audiokarma.www:001postfeed10220091004111818.gz:180,1278993897754 in 0sec
2010-07-15 16:22:35,254 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1254778440000:com.cartoondollemporium.www:001postfeed11120091007212511.gz:193,1278992096185
2010-07-15 16:22:35,256 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 147.6m; Skipped 4 files, size: 154810468
2010-07-15 16:22:35,257 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 81.4m; Skipped 2 files, size: 85387658
2010-07-15 16:22:35,257 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:35,258 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1254778440000:com.cartoondollemporium.www:001postfeed11120091007212511.gz:193,1278992096185 in 0sec
2010-07-15 16:22:35,258 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1254784140000:com.thumpertalk.www:001postfeed10720091006151052.gz:13,1278992096185
2010-07-15 16:22:35,260 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 120.6m; Skipped 3 files, size: 126460742
2010-07-15 16:22:35,261 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 65.4m; Skipped 3 files, size: 68522741
2010-07-15 16:22:35,261 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:35,262 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1254784140000:com.thumpertalk.www:001postfeed10720091006151052.gz:13,1278992096185 in 0sec
2010-07-15 16:22:35,262 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1254909960000:com.civinfo.www:002postfeed10420091007061627.gz:762,1278986327594
2010-07-15 16:22:35,264 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 178.7m; Skipped 4 files, size: 187407198
2010-07-15 16:22:35,265 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 106.0m; Skipped 4 files, size: 111126832
2010-07-15 16:22:35,265 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:35,266 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1254909960000:com.civinfo.www:002postfeed10420091007061627.gz:762,1278986327594 in 0sec
2010-07-15 16:22:35,266 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1254926580000:com.bodybuilding.www:002postfeed11120091007031814.gz:479,1278985947829
2010-07-15 16:22:35,268 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 175.2m; Skipped 2 file(s), size: 183373639
2010-07-15 16:22:35,268 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/74353742, seqid=25560872468
2010-07-15 16:22:35,370 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/74353742/core/7443901479284223401; store size is 175.2m
2010-07-15 16:22:35,371 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 101.3m; Skipped 2 file(s), size: 106063130
2010-07-15 16:22:35,371 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/74353742, seqid=25560872468
2010-07-15 16:22:35,471 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/74353742/tf/4486579767460358217; store size is 101.3m
2010-07-15 16:22:35,471 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:35,471 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1254926580000:com.bodybuilding.www:002postfeed11120091007031814.gz:479,1278985947829 in 0sec
2010-07-15 16:22:35,472 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1259593140000:net.bungie.www:005postfeed11220091201022110.gz:648,1279165410952
2010-07-15 16:22:35,473 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 182.2m; Skipped 2 file(s), size: 188834875
2010-07-15 16:22:35,473 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/698836005, seqid=25571419836
2010-07-15 16:22:35,782 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/698836005/core/4066682863016220081; store size is 182.3m
2010-07-15 16:22:35,783 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 103.7m; Skipped 2 file(s), size: 104039587
2010-07-15 16:22:35,783 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s)  into /hbase/feedData/compaction.dir/698836005, seqid=25571419836
2010-07-15 16:22:36,651 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/698836005/tf/5980641306568521064; store size is 103.7m
2010-07-15 16:22:36,651 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:36,652 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1259593140000:net.bungie.www:005postfeed11220091201022110.gz:648,1279165410952 in 1sec
2010-07-15 16:22:36,652 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258308540000:com.kaskus.www:001postfeed11020091116150051.gz:906,1279153717021
2010-07-15 16:22:36,655 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 218.9m; Skipped 3 files, size: 229045746
2010-07-15 16:22:36,656 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 121.4m; Skipped 3 files, size: 127013824
2010-07-15 16:22:36,656 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:36,657 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258308540000:com.kaskus.www:001postfeed11020091116150051.gz:906,1279153717021 in 0sec
2010-07-15 16:22:36,657 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1260113760000:com.softballfans.www:001postfeed10220091206232808.gz:589,1279168351865
2010-07-15 16:22:36,659 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 174.3m; Skipped 3 files, size: 180935346
2010-07-15 16:22:36,660 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 96.8m; Skipped 3 files, size: 100582336
2010-07-15 16:22:36,660 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:36,661 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1260113760000:com.softballfans.www:001postfeed10220091206232808.gz:589,1279168351865 in 0sec
2010-07-15 16:22:36,661 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258536180000:com.ligagame:001postfeed11120091118005034.gz:17,1279151867120
2010-07-15 16:22:36,665 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 231.5m; Skipped 2 files, size: 241550312
2010-07-15 16:22:36,667 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 139.0m; Skipped 1 file(s), size: 121373869
2010-07-15 16:22:36,667 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/974415518, seqid=25561101705
2010-07-15 16:22:40,847 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/974415518/tf/8399376528679961256; store size is 139.0m
2010-07-15 16:22:40,848 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:40,848 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258536180000:com.ligagame:001postfeed11120091118005034.gz:17,1279151867120 in 4sec
2010-07-15 16:22:40,849 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257236940000:net.trendinvest:002postfeed10420091103132039.gz:739,1279082200337
2010-07-15 16:22:40,850 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 241.4m; Skipped 2 file(s), size: 245696791
2010-07-15 16:22:40,851 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/843045542, seqid=25561101706
2010-07-15 16:22:41,726 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/843045542/core/5968738146183817062; store size is 241.4m
2010-07-15 16:22:41,727 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 146.0m; Skipped 2 file(s), size: 149001102
2010-07-15 16:22:41,728 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/843045542, seqid=25561101706
2010-07-15 16:22:42,909 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/843045542/tf/1576006512225147701; store size is 146.0m
2010-07-15 16:22:42,909 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:42,910 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257236940000:net.trendinvest:002postfeed10420091103132039.gz:739,1279082200337 in 2sec
2010-07-15 16:22:42,910 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1254873600000:com.thefashionspot.www:003postfeed10320091006211413.gz:443,1278988643057
2010-07-15 16:22:42,912 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 163.4m; Skipped 4 files, size: 171334556
2010-07-15 16:22:42,913 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 100.6m; Skipped 4 files, size: 105515757
2010-07-15 16:22:42,913 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:42,914 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1254873600000:com.thefashionspot.www:003postfeed10320091006211413.gz:443,1278988643057 in 0sec
2010-07-15 16:22:42,914 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258760340000:com.plentyoffish.www:002postfeed11120091121171731.gz:619,1279225537351
2010-07-15 16:22:42,916 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 129.6m; Skipped 2 files, size: 135038703
2010-07-15 16:22:42,917 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 77.2m; Skipped 2 files, size: 80425059
2010-07-15 16:22:42,917 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:42,918 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258760340000:com.plentyoffish.www:002postfeed11120091121171731.gz:619,1279225537351 in 0sec
2010-07-15 16:22:42,918 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1265505660000:com.gamescore.www:002postfeed10320100210042250.gz:134,1279224954787
2010-07-15 16:22:42,920 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 143.0m; Skipped 2 file(s), size: 143493844
2010-07-15 16:22:42,920 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/162771776, seqid=25561540273
2010-07-15 16:22:43,718 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/162771776/core/1763008378521949694; store size is 143.2m
2010-07-15 16:22:43,719 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 83.3m; Skipped 2 file(s), size: 84085851
2010-07-15 16:22:43,719 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/162771776, seqid=25561540273
2010-07-15 16:22:44,354 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/162771776/tf/4138108749494368752; store size is 83.3m
2010-07-15 16:22:44,354 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:44,354 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1265505660000:com.gamescore.www:002postfeed10320100210042250.gz:134,1279224954787 in 1sec
2010-07-15 16:22:44,355 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257505080000:com.facebook.www:003postfeed11220091107021731.gz:199,1279152150950
2010-07-15 16:22:44,356 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 195.1m; Skipped 3 files, size: 203622204
2010-07-15 16:22:44,357 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 116.6m; Skipped 2 file(s), size: 113796131
2010-07-15 16:22:44,357 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/426447386, seqid=25562078276
2010-07-15 16:22:45,901 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/426447386/tf/1048115160371177653; store size is 116.6m
2010-07-15 16:22:45,901 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:45,901 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257505080000:com.facebook.www:003postfeed11220091107021731.gz:199,1279152150950 in 1sec
2010-07-15 16:22:45,902 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257512100000:com.gametrailers.www:004postfeed11120091106183401.gz:454,1279150850046
2010-07-15 16:22:45,903 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 208.3m; Skipped 2 file(s), size: 203705502
2010-07-15 16:22:45,903 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/545286495, seqid=25562086735
2010-07-15 16:22:47,506 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/545286495/core/8797705421647185947; store size is 208.2m
2010-07-15 16:22:47,507 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 122.5m; Skipped 2 file(s), size: 120694948
2010-07-15 16:22:47,507 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/545286495, seqid=25562086735
2010-07-15 16:22:48,894 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/545286495/tf/1319318675892107455; store size is 122.5m
2010-07-15 16:22:48,895 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:48,895 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257512100000:com.gametrailers.www:004postfeed11120091106183401.gz:454,1279150850046 in 2sec
2010-07-15 16:22:48,895 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258187340000:com.extremefitness.www:002postfeed11220091115081507.gz:298,1279151804506
2010-07-15 16:22:48,897 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 218.0m; Skipped 2 files, size: 227346814
2010-07-15 16:22:48,898 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 129.8m; Skipped 3 files, size: 135317366
2010-07-15 16:22:48,898 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:48,899 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258187340000:com.extremefitness.www:002postfeed11220091115081507.gz:298,1279151804506 in 0sec
2010-07-15 16:22:48,899 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1259362860000:com.videogamereview.www:014postfeed10320091129122346.gz:267,1279168361923
2010-07-15 16:22:48,900 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 156.5m; Skipped 1 file(s), size: 140240566
2010-07-15 16:22:48,900 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/840684858, seqid=25562258860
2010-07-15 16:22:51,535 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/840684858/core/2291422191124088598; store size is 156.4m
2010-07-15 16:22:51,536 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 88.3m; Skipped 1 file(s), size: 80289688
2010-07-15 16:22:51,536 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/840684858, seqid=25562258860
2010-07-15 16:22:53,724 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/840684858/tf/9094124674748011400; store size is 88.3m
2010-07-15 16:22:53,724 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:53,726 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1259362860000:com.videogamereview.www:014postfeed10320091129122346.gz:267,1279168361923 in 4sec
2010-07-15 16:22:53,726 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1259402160000:com.macrumors.www:001postfeed10220091128191918.gz:26,1279165853641
2010-07-15 16:22:53,728 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 174.7m; Skipped 2 files, size: 182175573
2010-07-15 16:22:53,729 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 104.6m; Skipped 2 files, size: 109118601
2010-07-15 16:22:53,729 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:53,729 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1259402160000:com.macrumors.www:001postfeed10220091128191918.gz:26,1279165853641 in 0sec
2010-07-15 16:22:53,729 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1260192722000:com.yuku.shoguntotalwar:001postfeed10420091207091454.gz:113,1279164718749
2010-07-15 16:22:53,731 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 183.2m; Skipped 2 files, size: 190407076
2010-07-15 16:22:53,732 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 106.3m; Skipped 2 files, size: 110529346
2010-07-15 16:22:53,732 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:22:53,733 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1260192722000:com.yuku.shoguntotalwar:001postfeed10420091207091454.gz:113,1279164718749 in 0sec
2010-07-15 16:22:53,733 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258915560000:com.imdb:001postfeed10220091123020942.gz:873,1279167139349
2010-07-15 16:22:53,735 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 175.0m; Skipped 1 file(s), size: 153857673
2010-07-15 16:22:53,735 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1511515523, seqid=25563087792
2010-07-15 16:22:56,953 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1511515523/core/5399218584548991754; store size is 174.9m
2010-07-15 16:22:56,954 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 97.8m; Skipped 1 file(s), size: 86849640
2010-07-15 16:22:56,954 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1511515523, seqid=25563087792
2010-07-15 16:23:00,165 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1511515523/tf/2952822332118495953; store size is 97.8m
2010-07-15 16:23:00,165 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:00,166 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258915560000:com.imdb:001postfeed10220091123020942.gz:873,1279167139349 in 6sec
2010-07-15 16:23:00,166 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1263220860000:com.spacefem.www:012postfeed11020100111101658.gz:891,1279225914725
2010-07-15 16:23:00,168 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 136.4m; Skipped 2 file(s), size: 140631617
2010-07-15 16:23:00,168 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/1478693573, seqid=25569332242
2010-07-15 16:23:00,454 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1478693573/core/5299045085733901074; store size is 136.5m
2010-07-15 16:23:00,455 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 81.7m; Skipped 2 file(s), size: 84306907
2010-07-15 16:23:00,455 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/1478693573, seqid=25569332242
2010-07-15 16:23:00,721 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1478693573/tf/2510067830334989062; store size is 81.7m
2010-07-15 16:23:00,721 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:00,722 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1263220860000:com.spacefem.www:012postfeed11020100111101658.gz:891,1279225914725 in 0sec
2010-07-15 16:23:00,722 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1260036360000:com.gamespot.www:001postfeed10220091211030555.gz:219,1279164609855
2010-07-15 16:23:00,724 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 188.2m; Skipped 1 file(s), size: 164159665
2010-07-15 16:23:00,724 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1879316534, seqid=25563379404
2010-07-15 16:23:04,436 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1879316534/core/6278145365509623077; store size is 188.1m
2010-07-15 16:23:04,437 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 101.1m; Skipped 3 files, size: 104991611
2010-07-15 16:23:04,437 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:04,437 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1260036360000:com.gamespot.www:001postfeed10220091211030555.gz:219,1279164609855 in 3sec
2010-07-15 16:23:04,438 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1261526400000:com.toytowngermany.www:004postfeed11120091222200623.gz:162,1279160875001
2010-07-15 16:23:04,440 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 229.8m; Skipped 3 files, size: 239486107
2010-07-15 16:23:04,441 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 133.4m; Skipped 3 files, size: 138940910
2010-07-15 16:23:04,441 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:04,441 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1261526400000:com.toytowngermany.www:004postfeed11120091222200623.gz:162,1279160875001 in 0sec
2010-07-15 16:23:04,441 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258153980000:com.blu-ray.www:002postfeed11020091114140841.gz:119,1279151904163
2010-07-15 16:23:04,443 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 211.3m; Skipped 2 file(s), size: 213899975
2010-07-15 16:23:04,443 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/901622916, seqid=25563421642
2010-07-15 16:23:05,331 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/901622916/core/3935635152524726764; store size is 211.3m
2010-07-15 16:23:05,333 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 130.0m; Skipped 3 file(s), size: 134445668
2010-07-15 16:23:05,333 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/901622916, seqid=25563421642
2010-07-15 16:23:05,731 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/901622916/tf/7671248704694465138; store size is 130.0m
2010-07-15 16:23:05,731 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:05,732 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258153980000:com.blu-ray.www:002postfeed11020091114140841.gz:119,1279151904163 in 1sec
2010-07-15 16:23:05,732 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1260297120000:com.imdb:003postfeed10220091209010855.gz:42,1279163130655
2010-07-15 16:23:05,734 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 196.6m; Skipped 1 file(s), size: 169010066
2010-07-15 16:23:05,734 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1966478652, seqid=25563421644
2010-07-15 16:23:09,716 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1966478652/core/9053648770751669651; store size is 196.4m
2010-07-15 16:23:09,717 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 110.4m; Skipped 1 file(s), size: 96196171
2010-07-15 16:23:09,717 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1966478652, seqid=25563421644
2010-07-15 16:23:13,114 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1966478652/tf/282333901430794571; store size is 110.4m
2010-07-15 16:23:13,114 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:13,115 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1260297120000:com.imdb:003postfeed10220091209010855.gz:42,1279163130655 in 7sec
2010-07-15 16:23:13,115 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1259079900000:com.facebook.www:006postfeed11220091125012110.gz:495,1279168209415
2010-07-15 16:23:13,117 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 170.3m; Skipped 1 file(s), size: 150549380
2010-07-15 16:23:13,117 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/963080281, seqid=25563421645
2010-07-15 16:23:16,098 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/963080281/core/1781388223387070834; store size is 170.2m
2010-07-15 16:23:16,099 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 99.1m; Skipped 3 file(s), size: 101128797
2010-07-15 16:23:16,099 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/963080281, seqid=25563421645
2010-07-15 16:23:16,643 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/963080281/tf/7745802127366636981; store size is 99.1m
2010-07-15 16:23:16,644 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:16,644 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1259079900000:com.facebook.www:006postfeed11220091125012110.gz:495,1279168209415 in 3sec
2010-07-15 16:23:16,644 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258976700000:net.designcontest.www:001postfeed11120091124230914.gz:657,1279224049902
2010-07-15 16:23:16,646 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 143.0m; Skipped 3 file(s), size: 147170000
2010-07-15 16:23:16,646 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/2050365882, seqid=25563451298
2010-07-15 16:23:16,977 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2050365882/core/9102707118306905299; store size is 143.0m
2010-07-15 16:23:16,978 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 86.1m; Skipped 3 file(s), size: 88701090
2010-07-15 16:23:16,978 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/2050365882, seqid=25563451298
2010-07-15 16:23:17,291 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2050365882/tf/5189923986259265338; store size is 86.1m
2010-07-15 16:23:17,291 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:17,292 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258976700000:net.designcontest.www:001postfeed11120091124230914.gz:657,1279224049902 in 0sec
2010-07-15 16:23:17,292 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257943260000:com.steroidology.www:002postfeed11120091112131101.gz:212,1279148014024
2010-07-15 16:23:17,293 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 254.1m; Skipped 3 files, size: 265179129
2010-07-15 16:23:17,294 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 149.5m; Skipped 3 files, size: 156076939
2010-07-15 16:23:17,295 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:17,295 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257943260000:com.steroidology.www:002postfeed11120091112131101.gz:212,1279148014024 in 0sec
2010-07-15 16:23:17,295 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1259886960000:org.contour.www:002postfeed10420091205140559.gz:468,1279168426282
2010-07-15 16:23:17,297 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 160.3m; Skipped 1 file(s), size: 138778755
2010-07-15 16:23:17,297 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1702254153, seqid=25563995139
2010-07-15 16:23:20,525 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1702254153/core/7245009359824095090; store size is 160.2m
2010-07-15 16:23:20,526 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 91.6m; Skipped 1 file(s), size: 80649830
2010-07-15 16:23:20,526 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1702254153, seqid=25563995139
2010-07-15 16:23:23,422 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1702254153/tf/4995685475894443079; store size is 91.6m
2010-07-15 16:23:23,423 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:23,423 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1259886960000:org.contour.www:002postfeed10420091205140559.gz:468,1279168426282 in 6sec
2010-07-15 16:23:23,423 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257102300000:com.audizine.www:003postfeed10420091102220944.gz:663,1279124936502
2010-07-15 16:23:23,425 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 220.0m; Skipped 1 file(s), size: 194363598
2010-07-15 16:23:23,425 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1598260744, seqid=25564176360
2010-07-15 16:23:27,780 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1598260744/core/5770230037494250925; store size is 219.8m
2010-07-15 16:23:27,781 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 123.5m; Skipped 2 files, size: 129044359
2010-07-15 16:23:27,781 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:27,782 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257102300000:com.audizine.www:003postfeed10420091102220944.gz:663,1279124936502 in 4sec
2010-07-15 16:23:27,782 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1260935698000:com.yuku.theofficebodybuildingworkout28918:002postfeed10420091215231950.gz:264,1279226951088
2010-07-15 16:23:27,784 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 123.6m; Skipped 1 file(s), size: 124861564
2010-07-15 16:23:27,784 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/2045724338, seqid=25564394998
2010-07-15 16:23:28,458 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2045724338/core/2260173146058357469; store size is 123.7m
2010-07-15 16:23:28,459 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 76.6m; Skipped 1 file(s), size: 77842335
2010-07-15 16:23:28,459 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/2045724338, seqid=25564394998
2010-07-15 16:23:28,967 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2045724338/tf/7142232027295563794; store size is 76.6m
2010-07-15 16:23:28,967 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:28,968 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1260935698000:com.yuku.theofficebodybuildingworkout28918:002postfeed10420091215231950.gz:264,1279226951088 in 1sec
2010-07-15 16:23:28,968 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257272820000:com.explorerforum.www:002postfeed10420091104171000.gz:194,1279227016654
2010-07-15 16:23:28,970 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 122.3m; Skipped 2 files, size: 127785977
2010-07-15 16:23:28,971 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 73.1m; Skipped 2 files, size: 76423857
2010-07-15 16:23:28,971 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:28,972 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257272820000:com.explorerforum.www:002postfeed10420091104171000.gz:194,1279227016654 in 0sec
2010-07-15 16:23:28,972 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258671720000:com.bodybuilding.www:001postfeed11220091120103332.gz:425,1279150862524
2010-07-15 16:23:28,974 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 238.6m; Skipped 2 files, size: 249247501
2010-07-15 16:23:28,975 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 141.6m; Skipped 3 files, size: 147973644
2010-07-15 16:23:28,975 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:28,975 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258671720000:com.bodybuilding.www:001postfeed11220091120103332.gz:425,1279150862524 in 0sec
2010-07-15 16:23:28,975 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258904340000:net.bungie.www:002postfeed11220091123043825.gz:454,1279168122193
2010-07-15 16:23:28,977 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 162.8m; Skipped 1 file(s), size: 142177324
2010-07-15 16:23:28,977 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/2081694146, seqid=25564905625
2010-07-15 16:23:32,090 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2081694146/core/7838586004973680210; store size is 162.7m
2010-07-15 16:23:32,091 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 90.7m; Skipped 1 file(s), size: 80112241
2010-07-15 16:23:32,091 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/2081694146, seqid=25564905625
2010-07-15 16:23:34,550 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1528923, Hit=0, Miss=1528923, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN
2010-07-15 16:23:34,807 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2081694146/tf/4725234281121963964; store size is 90.7m
2010-07-15 16:23:34,807 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:34,807 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258904340000:net.bungie.www:002postfeed11220091123043825.gz:454,1279168122193 in 5sec
2010-07-15 16:23:34,808 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1261221300000:com.flickr.www:007postfeed11020091220000417.gz:69,1279225348954
2010-07-15 16:23:34,810 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 140.4m; Skipped 3 files, size: 146172541
2010-07-15 16:23:34,811 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 82.2m; Skipped 3 files, size: 85673389
2010-07-15 16:23:34,811 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:34,812 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1261221300000:com.flickr.www:007postfeed11020091220000417.gz:69,1279225348954 in 0sec
2010-07-15 16:23:34,812 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1256804760000:com.techsupportforum.www:001postfeed10220091101142039.gz:446,1279079944677
2010-07-15 16:23:34,814 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 230.7m; Skipped 4 files, size: 241278625
2010-07-15 16:23:34,815 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 140.0m; Skipped 3 files, size: 146385423
2010-07-15 16:23:34,815 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:34,816 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1256804760000:com.techsupportforum.www:001postfeed10220091101142039.gz:446,1279079944677 in 0sec
2010-07-15 16:23:34,816 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257452700000:org.sawmillcreek.www:002postfeed11120091107071015.gz:388,1279152114626
2010-07-15 16:23:34,820 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 202.3m; Skipped 1 file(s), size: 181474475
2010-07-15 16:23:34,820 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/2097542634, seqid=25565647739
2010-07-15 16:23:37,938 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2097542634/core/8475792916335438249; store size is 202.1m
2010-07-15 16:23:37,939 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 115.5m; Skipped 1 file(s), size: 104552769
2010-07-15 16:23:37,939 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/2097542634, seqid=25565647739
2010-07-15 16:23:40,851 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2097542634/tf/630647765513801898; store size is 115.5m
2010-07-15 16:23:40,851 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:40,852 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257452700000:org.sawmillcreek.www:002postfeed11120091107071015.gz:388,1279152114626 in 6sec
2010-07-15 16:23:40,852 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258235760000:com.immihelp.www:001postfeed11220091115045202.gz:511,1279154317340
2010-07-15 16:23:40,854 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 213.7m; Skipped 3 files, size: 223378555
2010-07-15 16:23:40,855 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 125.3m; Skipped 3 files, size: 131006140
2010-07-15 16:23:40,855 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:40,856 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258235760000:com.immihelp.www:001postfeed11220091115045202.gz:511,1279154317340 in 0sec
2010-07-15 16:23:40,856 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1262143440000:com.badmintoncentral.www:011postfeed11020100108120442.gz:771,1279225717407
2010-07-15 16:23:40,858 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 134.2m; Skipped 2 files, size: 140268777
2010-07-15 16:23:40,858 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 78.6m; Skipped 2 files, size: 82073333
2010-07-15 16:23:40,859 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:40,859 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1262143440000:com.badmintoncentral.www:011postfeed11020100108120442.gz:771,1279225717407 in 0sec
2010-07-15 16:23:40,859 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1261312860000:com.driftworks.www:001postfeed10320091220100730.gz:933,1279162348017
2010-07-15 16:23:40,861 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 228.3m; Skipped 2 file(s), size: 226110492
2010-07-15 16:23:40,861 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/90437576, seqid=25565999163
2010-07-15 16:23:42,347 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/90437576/core/3622218594784556225; store size is 228.5m
2010-07-15 16:23:42,348 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 127.1m; Skipped 2 file(s), size: 126421934
2010-07-15 16:23:42,348 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/90437576, seqid=25565999163
2010-07-15 16:23:43,591 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/90437576/tf/6140840426521464833; store size is 127.1m
2010-07-15 16:23:43,591 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:43,592 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1261312860000:com.driftworks.www:001postfeed10320091220100730.gz:933,1279162348017 in 2sec
2010-07-15 16:23:43,592 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954
2010-07-15 16:23:43,594 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 257.0m; Skipped 2 file(s), size: 254284485
2010-07-15 16:23:43,594 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1855822307, seqid=25565999165
2010-07-15 16:23:45,236 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1855822307/core/7649881314149793876; store size is 257.0m
2010-07-15 16:23:45,237 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 146.0m; Skipped 2 file(s), size: 144884656
2010-07-15 16:23:45,237 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1855822307, seqid=25565999165
2010-07-15 16:23:46,675 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1855822307/tf/1374900261190937036; store size is 146.0m
2010-07-15 16:23:46,675 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:46,676 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954 in 3sec
2010-07-15 16:23:46,676 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting split of region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954
2010-07-15 16:23:46,677 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954: compactions & flushes disabled
2010-07-15 16:23:46,677 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region, no outstanding scanners on feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954
2010-07-15 16:23:46,677 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954
2010-07-15 16:23:46,678 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954. Current region memstore size 4.3m
2010-07-15 16:23:46,703 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1855822307/core/941292609318344827, entries=7873, sequenceid=25573807453, memsize=2.0m, filesize=243.0k to feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954
2010-07-15 16:23:46,726 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1855822307/tf/6071270689075181248, entries=11464, sequenceid=25573807453, memsize=2.4m, filesize=137.2k to feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954
2010-07-15 16:23:46,726 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~4.3m for region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954 in 49ms, sequence id=25573807453, compaction requested=true
2010-07-15 16:23:46,726 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed core
2010-07-15 16:23:46,727 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed tf
2010-07-15 16:23:46,727 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed tfidf
2010-07-15 16:23:46,727 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954
2010-07-15 16:23:46,858 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676, encoded=1648690824
2010-07-15 16:23:46,955 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/core/3970717163955321862.1855822307, isReference=true, sequence id=25069896401, length=223531208, majorCompaction=true
2010-07-15 16:23:46,960 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/core/7649881314149793876.1855822307, isReference=true, sequence id=25565999165, length=15188201, majorCompaction=false
2010-07-15 16:23:46,996 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/core/8978986931285315329.1855822307, isReference=true, sequence id=25445171115, length=30753277, majorCompaction=false
2010-07-15 16:23:47,001 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/core/941292609318344827.1855822307, isReference=true, sequence id=25573807453, length=248853, majorCompaction=false
2010-07-15 16:23:47,006 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/tf/1374900261190937036.1855822307, isReference=true, sequence id=25565999165, length=8169813, majorCompaction=false
2010-07-15 16:23:47,034 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/tf/5688947589019900690.1855822307, isReference=true, sequence id=25445171115, length=15942323, majorCompaction=false
2010-07-15 16:23:47,085 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/tf/5717940824201118411.1855822307, isReference=true, sequence id=25069896401, length=128942333, majorCompaction=true
2010-07-15 16:23:47,089 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/tf/6071270689075181248.1855822307, isReference=true, sequence id=25573807453, length=140511, majorCompaction=false
2010-07-15 16:23:47,091 INFO org.apache.hadoop.hbase.regionserver.HRegion: region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676/1648690824 available; sequence id is 25573807454
2010-07-15 16:23:47,091 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676: compactions & flushes disabled
2010-07-15 16:23:47,091 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region, no outstanding scanners on feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676
2010-07-15 16:23:47,091 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676
2010-07-15 16:23:47,091 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed core
2010-07-15 16:23:47,091 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed tf
2010-07-15 16:23:47,091 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed tfidf
2010-07-15 16:23:47,091 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676
2010-07-15 16:23:47,091 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676, encoded=51680204
2010-07-15 16:23:47,121 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/core/3970717163955321862.1855822307, isReference=true, sequence id=25069896402, length=223531208, majorCompaction=true
2010-07-15 16:23:47,125 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/core/7649881314149793876.1855822307, isReference=true, sequence id=25565999166, length=15188201, majorCompaction=false
2010-07-15 16:23:47,131 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/core/8978986931285315329.1855822307, isReference=true, sequence id=25445171116, length=30753277, majorCompaction=false
2010-07-15 16:23:47,134 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/core/941292609318344827.1855822307, isReference=true, sequence id=25573807454, length=248853, majorCompaction=false
2010-07-15 16:23:47,140 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/tf/1374900261190937036.1855822307, isReference=true, sequence id=25565999166, length=8169813, majorCompaction=false
2010-07-15 16:23:47,145 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/tf/5688947589019900690.1855822307, isReference=true, sequence id=25445171116, length=15942323, majorCompaction=false
2010-07-15 16:23:47,167 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/tf/5717940824201118411.1855822307, isReference=true, sequence id=25069896402, length=128942333, majorCompaction=true
2010-07-15 16:23:47,170 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/tf/6071270689075181248.1855822307, isReference=true, sequence id=25573807454, length=140511, majorCompaction=false
2010-07-15 16:23:47,172 INFO org.apache.hadoop.hbase.regionserver.HRegion: region feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676/51680204 available; sequence id is 25573807455
2010-07-15 16:23:47,172 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676: compactions & flushes disabled
2010-07-15 16:23:47,172 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region, no outstanding scanners on feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676
2010-07-15 16:23:47,172 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676
2010-07-15 16:23:47,172 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed core
2010-07-15 16:23:47,172 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed tf
2010-07-15 16:23:47,173 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed tfidf
2010-07-15 16:23:47,173 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676
2010-07-15 16:23:47,173 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Cleaned up /hbase/feedData/1855822307/splits true
2010-07-15 16:23:47,176 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: region split, META updated, and report to master all successful. Old region=REGION => {NAME => 'feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279150984954', STARTKEY => '1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807', ENDKEY => '1258739880000:com.espn:004postfeed11020091120171422.gz:67', ENCODED => 1855822307, OFFLINE => true, SPLIT => true, TABLE => {{NAME => 'feedData', FAMILIES => [{NAME => 'core', VERSIONS => '3', COMPRESSION => 'LZO', TTL => '2147483647', BLOCKSIZE => '131072', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'tf', VERSIONS => '3', COMPRESSION => 'LZO', TTL => '2147483647', BLOCKSIZE => '131072', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'tfidf', VERSIONS => '3', COMPRESSION => 'LZO', TTL => '2147483647', BLOCKSIZE => '131072', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}, new regions: feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676, feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676. Split took 0sec
2010-07-15 16:23:47,176 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257859740000:com.steampowered.www:004postfeed10320091110181336.gz:48,1279147200956
2010-07-15 16:23:47,178 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 245.0m; Skipped 4 files, size: 255437693
2010-07-15 16:23:47,179 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 146.4m; Skipped 2 files, size: 152495090
2010-07-15 16:23:47,179 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:47,179 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257859740000:com.steampowered.www:004postfeed10320091110181336.gz:48,1279147200956 in 0sec
2010-07-15 16:23:47,180 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1259944980000:com.city-data.www:002postfeed10320091204010340.gz:172,1279168751812
2010-07-15 16:23:47,181 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 171.3m; Skipped 2 file(s), size: 176970697
2010-07-15 16:23:47,181 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/1759064528, seqid=25571419838
2010-07-15 16:23:47,484 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1759064528/core/5461059706694749057; store size is 171.4m
2010-07-15 16:23:47,485 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 95.0m; Skipped 3 file(s), size: 98125665
2010-07-15 16:23:47,485 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/1759064528, seqid=25571419838
2010-07-15 16:23:47,751 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1759064528/tf/7220694602498851537; store size is 95.0m
2010-07-15 16:23:47,751 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:47,752 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1259944980000:com.city-data.www:002postfeed10320091204010340.gz:172,1279168751812 in 0sec
2010-07-15 16:23:47,752 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1256010780000:com.kaskus.www:001postfeed11220091025004719.gz:301,1279123324227
2010-07-15 16:23:47,753 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 162.5m; Skipped 3 file(s), size: 169433511
2010-07-15 16:23:47,754 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/830914069, seqid=25567284059
2010-07-15 16:23:47,883 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/830914069/core/6573078483419996779; store size is 162.5m
2010-07-15 16:23:47,884 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 97.2m; Skipped 3 file(s), size: 101377919
2010-07-15 16:23:47,884 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/830914069, seqid=25567284059
2010-07-15 16:23:48,070 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/830914069/tf/4601585658388920775; store size is 97.2m
2010-07-15 16:23:48,070 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:48,071 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1256010780000:com.kaskus.www:001postfeed11220091025004719.gz:301,1279123324227 in 0sec
2010-07-15 16:23:48,071 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1256981040000:com.gamefaqs.www:006postfeed11020091103050222.gz:369,1279154379130
2010-07-15 16:23:48,073 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 172.2m; Skipped 4 files, size: 180181502
2010-07-15 16:23:48,074 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 100.5m; Skipped 4 files, size: 105206422
2010-07-15 16:23:48,074 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:48,074 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1256981040000:com.gamefaqs.www:006postfeed11020091103050222.gz:369,1279154379130 in 0sec
2010-07-15 16:23:48,074 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1260745200000:com.imdb.www:001postfeed10420091214101632.gz:427,1279227318950
2010-07-15 16:23:48,076 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 121.0m; Skipped 2 files, size: 126237846
2010-07-15 16:23:48,077 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 78.2m; Skipped 2 files, size: 81672953
2010-07-15 16:23:48,077 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:48,078 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1260745200000:com.imdb.www:001postfeed10420091214101632.gz:427,1279227318950 in 0sec
2010-07-15 16:23:48,078 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1262439000000:com.civfanatics.www:001postfeed11120100107133950.gz:368,1279169126647
2010-07-15 16:23:48,080 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 170.7m; Skipped 2 file(s), size: 169253050
2010-07-15 16:23:48,080 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1032744068, seqid=25567660560
2010-07-15 16:23:48,419 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676
2010-07-15 16:23:48,419 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676
2010-07-15 16:23:48,419 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN: feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676
2010-07-15 16:23:48,420 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676, encoded=51680204
2010-07-15 16:23:48,446 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/core/3970717163955321862.1855822307, isReference=true, sequence id=25069896402, length=223531208, majorCompaction=true
2010-07-15 16:23:48,451 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/core/7649881314149793876.1855822307, isReference=true, sequence id=25565999166, length=15188201, majorCompaction=false
2010-07-15 16:23:48,457 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/core/8978986931285315329.1855822307, isReference=true, sequence id=25445171116, length=30753277, majorCompaction=false
2010-07-15 16:23:48,461 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/core/941292609318344827.1855822307, isReference=true, sequence id=25573807454, length=248853, majorCompaction=false
2010-07-15 16:23:48,467 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/tf/1374900261190937036.1855822307, isReference=true, sequence id=25565999166, length=8169813, majorCompaction=false
2010-07-15 16:23:48,472 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/tf/5688947589019900690.1855822307, isReference=true, sequence id=25445171116, length=15942323, majorCompaction=false
2010-07-15 16:23:48,494 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/tf/5717940824201118411.1855822307, isReference=true, sequence id=25069896402, length=128942333, majorCompaction=true
2010-07-15 16:23:48,498 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/51680204/tf/6071270689075181248.1855822307, isReference=true, sequence id=25573807454, length=140511, majorCompaction=false
2010-07-15 16:23:48,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: region feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676/51680204 available; sequence id is 25573807455
2010-07-15 16:23:48,500 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676/51680204 because: Region has references on open
2010-07-15 16:23:48,500 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN: feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676
2010-07-15 16:23:48,501 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676, encoded=1648690824
2010-07-15 16:23:48,527 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/core/3970717163955321862.1855822307, isReference=true, sequence id=25069896401, length=223531208, majorCompaction=true
2010-07-15 16:23:48,531 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/core/7649881314149793876.1855822307, isReference=true, sequence id=25565999165, length=15188201, majorCompaction=false
2010-07-15 16:23:48,537 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/core/8978986931285315329.1855822307, isReference=true, sequence id=25445171115, length=30753277, majorCompaction=false
2010-07-15 16:23:48,541 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/core/941292609318344827.1855822307, isReference=true, sequence id=25573807453, length=248853, majorCompaction=false
2010-07-15 16:23:48,547 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/tf/1374900261190937036.1855822307, isReference=true, sequence id=25565999165, length=8169813, majorCompaction=false
2010-07-15 16:23:48,553 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/tf/5688947589019900690.1855822307, isReference=true, sequence id=25445171115, length=15942323, majorCompaction=false
2010-07-15 16:23:48,574 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/tf/5717940824201118411.1855822307, isReference=true, sequence id=25069896401, length=128942333, majorCompaction=true
2010-07-15 16:23:48,577 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hbase/feedData/1648690824/tf/6071270689075181248.1855822307, isReference=true, sequence id=25573807453, length=140511, majorCompaction=false
2010-07-15 16:23:48,580 INFO org.apache.hadoop.hbase.regionserver.HRegion: region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676/1648690824 available; sequence id is 25573807454
2010-07-15 16:23:48,580 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676/1648690824 because: Region has references on open
2010-07-15 16:23:49,284 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1032744068/core/3417764739701916849; store size is 170.8m
2010-07-15 16:23:49,285 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 104.3m; Skipped 2 file(s), size: 103629825
2010-07-15 16:23:49,285 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1032744068, seqid=25567660560
2010-07-15 16:23:50,140 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1032744068/tf/3311070922583630761; store size is 104.3m
2010-07-15 16:23:50,140 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:50,141 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1262439000000:com.civfanatics.www:001postfeed11120100107133950.gz:368,1279169126647 in 2sec
2010-07-15 16:23:50,141 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258470900000:com.blackvoices.www:001postfeed10320091118120233.gz:357,1279152670998
2010-07-15 16:23:50,142 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 233.1m; Skipped 1 file(s), size: 201591533
2010-07-15 16:23:50,142 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/999904182, seqid=25567879186
2010-07-15 16:23:54,240 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/999904182/core/4718628922527978548; store size is 233.0m
2010-07-15 16:23:54,242 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 135.8m; Skipped 3 files, size: 142026834
2010-07-15 16:23:54,242 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:54,242 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258470900000:com.blackvoices.www:001postfeed10320091118120233.gz:357,1279152670998 in 4sec
2010-07-15 16:23:54,242 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1262159760000:com.giantitp.www:006postfeed11020100106023136.gz:754,1279225717407
2010-07-15 16:23:54,244 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 134.9m; Skipped 2 files, size: 141355747
2010-07-15 16:23:54,245 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 78.7m; Skipped 2 files, size: 82443201
2010-07-15 16:23:54,246 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:54,246 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1262159760000:com.giantitp.www:006postfeed11020100106023136.gz:754,1279225717407 in 0sec
2010-07-15 16:23:54,246 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258771500000:com.bt3central:003postfeed11020091121060335.gz:179,1279225537351
2010-07-15 16:23:54,248 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 135.7m; Skipped 2 files, size: 142006714
2010-07-15 16:23:54,249 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 79.6m; Skipped 2 files, size: 83289975
2010-07-15 16:23:54,250 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:54,250 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258771500000:com.bt3central:003postfeed11020091121060335.gz:179,1279225537351 in 0sec
2010-07-15 16:23:54,250 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1259817360000:com.videogamereview.www:013postfeed10320091205095719.gz:266,1279162565600
2010-07-15 16:23:54,252 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 215.5m; Skipped 2 file(s), size: 217075882
2010-07-15 16:23:54,252 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/850819095, seqid=25568050780
2010-07-15 16:23:55,083 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/850819095/core/6104979127321579817; store size is 215.6m
2010-07-15 16:23:55,085 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 126.3m; Skipped 3 files, size: 131817514
2010-07-15 16:23:55,085 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:55,086 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1259817360000:com.videogamereview.www:013postfeed10320091205095719.gz:266,1279162565600 in 0sec
2010-07-15 16:23:55,086 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1264259160000:com.imdb.www:002postfeed10420100124031507.gz:242,1279171186846
2010-07-15 16:23:55,087 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 163.4m; Skipped 3 files, size: 169957452
2010-07-15 16:23:55,088 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 95.6m; Skipped 3 files, size: 99656292
2010-07-15 16:23:55,088 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:55,089 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1264259160000:com.imdb.www:002postfeed10420100124031507.gz:242,1279171186846 in 0sec
2010-07-15 16:23:55,089 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1254713340000:com.imdb.www:003postfeed11120091012220432.gz:126,1278992950314
2010-07-15 16:23:55,091 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 125.9m; Skipped 2 file(s), size: 131903877
2010-07-15 16:23:55,091 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/938553351, seqid=25568752829
2010-07-15 16:23:55,163 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/938553351/core/4099673044287172786; store size is 125.9m
2010-07-15 16:23:55,164 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 74.8m; Skipped 2 file(s), size: 78427627
2010-07-15 16:23:55,164 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/938553351, seqid=25568752829
2010-07-15 16:23:55,219 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/938553351/tf/6648530044771233626; store size is 74.8m
2010-07-15 16:23:55,219 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:55,220 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1254713340000:com.imdb.www:003postfeed11120091012220432.gz:126,1278992950314 in 0sec
2010-07-15 16:23:55,220 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257837660000:com.imdb:001postfeed10220091110231051.gz:806,1279148072076
2010-07-15 16:23:55,221 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 244.8m; Skipped 3 files, size: 255491434
2010-07-15 16:23:55,222 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 149.9m; Skipped 3 files, size: 156482528
2010-07-15 16:23:55,223 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:55,225 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257837660000:com.imdb:001postfeed10220091110231051.gz:806,1279148072076 in 0sec
2010-07-15 16:23:55,225 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1260472320000:com.seriouseats.www:001postfeed10320091211070631.gz:205,1279163196909
2010-07-15 16:23:55,227 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 204.7m; Skipped 3 files, size: 213127016
2010-07-15 16:23:55,228 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 113.5m; Skipped 2 files, size: 118230359
2010-07-15 16:23:55,228 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:55,228 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1260472320000:com.seriouseats.www:001postfeed10320091211070631.gz:205,1279163196909 in 0sec
2010-07-15 16:23:55,229 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1263242400000:com.animesuki.www:001postfeed11120100112190335.gz:76,1279225914725
2010-07-15 16:23:55,230 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 134.2m; Skipped 2 file(s), size: 132698486
2010-07-15 16:23:55,230 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/226854815, seqid=25569332239
2010-07-15 16:23:56,197 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/226854815/core/2607985940837122672; store size is 134.3m
2010-07-15 16:23:56,198 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 77.7m; Skipped 2 files, size: 80810855
2010-07-15 16:23:56,198 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:56,199 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1263242400000:com.animesuki.www:001postfeed11120100112190335.gz:76,1279225914725 in 0sec
2010-07-15 16:23:56,199 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1264103040000:com.hardforum.www:003postfeed10720100121230943.gz:49,1279161283437
2010-07-15 16:23:56,200 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 227.2m; Skipped 3 files, size: 236442306
2010-07-15 16:23:56,201 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 132.7m; Skipped 3 files, size: 138043316
2010-07-15 16:23:56,201 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:56,202 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1264103040000:com.hardforum.www:003postfeed10720100121230943.gz:49,1279161283437 in 0sec
2010-07-15 16:23:56,202 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257868800000:com.imdb.www:003postfeed10720091112030734.gz:370,1279147200956
2010-07-15 16:23:56,204 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 252.4m; Skipped 3 files, size: 263011128
2010-07-15 16:23:56,205 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 148.6m; Skipped 4 files, size: 154830000
2010-07-15 16:23:56,205 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:56,205 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257868800000:com.imdb.www:003postfeed10720091112030734.gz:370,1279147200956 in 0sec
2010-07-15 16:23:56,206 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1254654900000:com.clubarmada.www:003postfeed11120091006150250.gz:874,1278992690896
2010-07-15 16:23:56,207 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 132.9m; Skipped 4 file(s), size: 139316915
2010-07-15 16:23:56,207 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/472525135, seqid=25569672671
2010-07-15 16:23:56,253 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/472525135/core/1243907190692554501; store size is 132.9m
2010-07-15 16:23:56,254 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 75.4m; Skipped 3 file(s), size: 79043030
2010-07-15 16:23:56,254 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/472525135, seqid=25569672671
2010-07-15 16:23:56,292 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/472525135/tf/1383629494108452842; store size is 75.4m
2010-07-15 16:23:56,292 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:56,293 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1254654900000:com.clubarmada.www:003postfeed11120091006150250.gz:874,1278992690896 in 0sec
2010-07-15 16:23:56,293 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257728400000:com.apple.www:002postfeed11020091109061130.gz:57,1279149851181
2010-07-15 16:23:56,295 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 238.2m; Skipped 3 files, size: 248765892
2010-07-15 16:23:56,296 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 146.9m; Skipped 3 files, size: 153413098
2010-07-15 16:23:56,296 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:56,297 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257728400000:com.apple.www:002postfeed11020091109061130.gz:57,1279149851181 in 0sec
2010-07-15 16:23:56,297 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1254315780000:com.wetcanvas.www:002postfeed11120091001040501.gz:817,1278992101211
2010-07-15 16:23:56,299 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 195.6m; Skipped 2 file(s), size: 204890813
2010-07-15 16:23:56,299 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/816622108, seqid=25571109519
2010-07-15 16:23:56,367 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/816622108/core/7641614799026281073; store size is 195.6m
2010-07-15 16:23:56,368 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 112.1m; Skipped 2 file(s), size: 117421012
2010-07-15 16:23:56,368 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/816622108, seqid=25571109519
2010-07-15 16:23:56,433 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/816622108/tf/4888088848590038962; store size is 112.1m
2010-07-15 16:23:56,433 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:56,434 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1254315780000:com.wetcanvas.www:002postfeed11120091001040501.gz:817,1278992101211 in 0sec
2010-07-15 16:23:56,434 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258605780000:com.afghanforums.www:004postfeed11120091120061446.gz:396,1279152732292
2010-07-15 16:23:56,436 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 235.1m; Skipped 2 file(s), size: 237114466
2010-07-15 16:23:56,436 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/429295832, seqid=25571292541
2010-07-15 16:23:57,339 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/429295832/core/8674515850146456431; store size is 235.1m
2010-07-15 16:23:57,340 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 139.6m; Skipped 2 file(s), size: 141384180
2010-07-15 16:23:57,340 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/429295832, seqid=25571292541
2010-07-15 16:23:58,477 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/429295832/tf/3342246663175576066; store size is 139.6m
2010-07-15 16:23:58,478 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:58,478 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258605780000:com.afghanforums.www:004postfeed11120091120061446.gz:396,1279152732292 in 2sec
2010-07-15 16:23:58,478 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1259434020000:com.gamefaqs.www:001postfeed11020091129170204.gz:35,1279166206994
2010-07-15 16:23:58,480 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 175.6m; Skipped 2 file(s), size: 179336200
2010-07-15 16:23:58,480 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s)  into /hbase/feedData/compaction.dir/1003205351, seqid=25571419832
2010-07-15 16:23:58,931 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1003205351/core/8283708655513829356; store size is 175.6m
2010-07-15 16:23:58,932 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 97.4m; Skipped 3 files, size: 101404321
2010-07-15 16:23:58,932 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:58,933 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1259434020000:com.gamefaqs.www:001postfeed11020091129170204.gz:35,1279166206994 in 0sec
2010-07-15 16:23:58,933 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1261194000000:org.d2jsp.www:002postfeed11120091219040810.gz:677,1279227945119
2010-07-15 16:23:58,934 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:23:58,934 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1261194000000:org.d2jsp.www:002postfeed11120091219040810.gz:677,1279227945119 in 0sec
2010-07-15 16:23:58,934 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257292800000:com.forumgarden.www:009postfeed10320091104172418.gz:662,1279223937290
2010-07-15 16:23:58,936 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 136.3m; Skipped 3 files, size: 142402498
2010-07-15 16:23:58,937 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 84.4m; Skipped 1 file(s), size: 79318172
2010-07-15 16:23:58,937 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/2127686503, seqid=25571731791
2010-07-15 16:24:00,298 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/2127686503/tf/2480999255970918126; store size is 84.4m
2010-07-15 16:24:00,298 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:24:00,299 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257292800000:com.forumgarden.www:009postfeed10320091104172418.gz:662,1279223937290 in 1sec
2010-07-15 16:24:00,299 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1255274100000:com.fullsizebronco.www:001postfeed10220091012040326.gz:707,1278986000848
2010-07-15 16:24:00,301 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 177.1m; Skipped 2 files, size: 185676657
2010-07-15 16:24:00,302 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 98.9m; Skipped 2 files, size: 103727882
2010-07-15 16:24:00,302 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:24:00,303 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1255274100000:com.fullsizebronco.www:001postfeed10220091012040326.gz:707,1278986000848 in 0sec
2010-07-15 16:24:00,303 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1255975500000:com.kirupa.www:002postfeed10720091021040430.gz:897,1279146010516
2010-07-15 16:24:00,304 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 160.2m; Skipped 2 file(s), size: 160948768
2010-07-15 16:24:00,304 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/791457190, seqid=25572194661
2010-07-15 16:24:00,958 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/791457190/core/4578190864726255233; store size is 160.2m
2010-07-15 16:24:00,959 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 90.5m; Skipped 2 file(s), size: 91173656
2010-07-15 16:24:00,959 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/791457190, seqid=25572194661
2010-07-15 16:24:01,511 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/791457190/tf/5273864396858721836; store size is 90.5m
2010-07-15 16:24:01,511 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:24:01,512 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1255975500000:com.kirupa.www:002postfeed10720091021040430.gz:897,1279146010516 in 1sec
2010-07-15 16:24:01,512 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1255988880000:com.mac-forums.www:001postfeed10720091020040208.gz:382,1279124086728
2010-07-15 16:24:01,514 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 163.5m; Skipped 3 files, size: 171358759
2010-07-15 16:24:01,515 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 92.2m; Skipped 4 files, size: 96629857
2010-07-15 16:24:01,515 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:24:01,516 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1255988880000:com.mac-forums.www:001postfeed10720091020040208.gz:382,1279124086728 in 0sec
2010-07-15 16:24:01,516 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1255560373000:com.tv.www:001postfeed10720091015000519.gz:3,1278983238206
2010-07-15 16:24:01,518 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of core: 203.0m; Skipped 3 files, size: 212798121
2010-07-15 16:24:01,519 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of tf: 118.8m; Skipped 3 files, size: 124517240
2010-07-15 16:24:01,519 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:24:01,520 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1255560373000:com.tv.www:001postfeed10720091015000519.gz:3,1278983238206 in 0sec
2010-07-15 16:24:01,520 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1255438260000:com.madjacksports.www:002postfeed11120091014032517.gz:398,1279082417875
2010-07-15 16:24:01,521 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 146.1m; Skipped 2 file(s), size: 149931876
2010-07-15 16:24:01,521 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/95413503, seqid=25572937069
2010-07-15 16:24:01,943 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/95413503/core/949475209750832805; store size is 146.1m
2010-07-15 16:24:01,944 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 86.1m; Skipped 2 file(s), size: 88422916
2010-07-15 16:24:01,944 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/95413503, seqid=25572937069
2010-07-15 16:24:02,260 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/95413503/tf/5950582710867524798; store size is 86.1m
2010-07-15 16:24:02,260 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:24:02,261 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1255438260000:com.madjacksports.www:002postfeed11120091014032517.gz:398,1279082417875 in 0sec
2010-07-15 16:24:02,261 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1257374940000:com.pbnation.www:002postfeed10320091106220606.gz:805,1279147145223
2010-07-15 16:24:02,263 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 221.3m; Skipped 2 file(s), size: 214830720
2010-07-15 16:24:02,263 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1571341242, seqid=25573102895
2010-07-15 16:24:03,931 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1571341242/core/1244322437612392608; store size is 221.3m
2010-07-15 16:24:03,932 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 135.5m; Skipped 2 file(s), size: 132310130
2010-07-15 16:24:03,933 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1571341242, seqid=25573102895
2010-07-15 16:24:05,353 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1571341242/tf/6614177082989852047; store size is 135.5m
2010-07-15 16:24:05,354 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:24:05,354 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1257374940000:com.pbnation.www:002postfeed10320091106220606.gz:805,1279147145223 in 3sec
2010-07-15 16:24:05,354 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1256076180000:com.flyertalk.www:002postfeed10320091021081252.gz:281,1279155251907
2010-07-15 16:24:05,356 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of core: 164.3m; Skipped 2 file(s), size: 166036886
2010-07-15 16:24:05,356 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1711168961, seqid=25573339886
2010-07-15 16:24:05,947 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1711168961/core/9094011625996502737; store size is 164.3m
2010-07-15 16:24:05,948 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of tf: 93.1m; Skipped 4 file(s), size: 96779823
2010-07-15 16:24:05,948 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s)  into /hbase/feedData/compaction.dir/1711168961, seqid=25573339886
2010-07-15 16:24:06,150 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1711168961/tf/6791208876160978782; store size is 93.1m
2010-07-15 16:24:06,151 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:24:06,151 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1256076180000:com.flyertalk.www:002postfeed10320091021081252.gz:281,1279155251907 in 0sec
2010-07-15 16:24:06,151 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676
2010-07-15 16:24:06,153 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hbase/feedData/compaction.dir/51680204, seqid=25573807454
2010-07-15 16:24:22,512 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/51680204/core/7287973117914311678; store size is 116.2m
2010-07-15 16:24:22,513 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hbase/feedData/compaction.dir/51680204, seqid=25573807454
2010-07-15 16:24:34,549 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1548704, Hit=0, Miss=1548704, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN
2010-07-15 16:24:34,658 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/51680204/tf/5793440643968733547; store size is 69.9m
2010-07-15 16:24:34,658 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:24:34,659 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258732260000:com.playstation.us.www:001postfeed10720091121020533.gz:227,1279229026676 in 28sec
2010-07-15 16:24:34,659 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676
2010-07-15 16:24:34,660 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hbase/feedData/compaction.dir/1648690824, seqid=25573807453
2010-07-15 16:24:48,172 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of core; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1648690824/core/1180942171402887617; store size is 112.8m
2010-07-15 16:24:48,173 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hbase/feedData/compaction.dir/1648690824, seqid=25573807453
2010-07-15 16:25:04,026 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of tf; new storefile is hdfs://nn01.colo.networkedinsights.com:8020/hbase/feedData/1648690824/tf/3393505379998291511; store size is 67.4m
2010-07-15 16:25:04,026 DEBUG org.apache.hadoop.hbase.regionserver.Store: tfidf: no store files to compact
2010-07-15 16:25:04,027 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region feedData,1258725660000:com.gaiaonline.www:002postfeed11020091121040404.gz:807,1279229026676 in 29sec
2010-07-15 16:25:34,549 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1558249, Hit=0, Miss=1558249, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN
2010-07-15 16:26:34,549 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1558249, Hit=0, Miss=1558249, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN
2010-07-15 16:27:34,549 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1558249, Hit=0, Miss=1558249, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN
2010-07-15 16:28:34,549 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1558249, Hit=0, Miss=1558249, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN
2010-07-15 16:29:34,549 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1558249, Hit=0, Miss=1558249, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN
2010-07-15 16:30:34,549 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1558249, Hit=0, Miss=1558249, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN
2010-07-15 16:31:34,549 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=6.519417MB (6836104), Free=788.0806MB (826362424), Max=794.60004MB (833198528), Counts: Blocks=0, Access=1558249, Hit=0, Miss=1558249, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN


On 7/15/10 9:10 PM, "Luke Forehand" <luke.forehand@networkedinsights.com> wrote:


I have attached a portion of the log file, from when it was compacting and flushing normally, until the region server stopped doing anything but spewing stats.


On 7/15/10 6:30 PM, "Jean-Daniel Cryans" <jdcryans@apache.org> wrote:

Nothing particular in that dump, and I'm not aware of any deadlock in
0.20.3, could we see the region server log?

Thx

J-D

On Thu, Jul 15, 2010 at 4:17 PM, Luke Forehand
<luke.forehand@networkedinsights.com> wrote:
> First post evar!
>
> I have a 3 node cluster and have set mapred.tasktracker.map.tasks.maximum=4.  I am using Cloudera's CDH2 dist (hadoop 0.20.1 and hbase 0.20.3).  I am importing a 15GB sequence file into an HTable instance straight from the Mapper, no reduce phase.  At random times during the job, the RegionServer JVMs start to completely hang.  I started up jstatd on the servers and used VisualVM to profile the processes.  When I encounter the hang, I notice absolutely no problems with heap or garbage collection, the region servers just become vegetables within minutes of eachother.  This fact, along with the randomness of the hang, leads me to believe there is a dead lock due to a race condition somewhere in the code.  I suspect something similar to the following issues:
>
> http://issues.apache.org/jira/browse/HADOOP-5859
> http://issues.apache.org/jira/browse/HBASE-667
>
> Here is a thread dump of one of the region server procs immediately after it went silent.  Would anybody that has more knowledge about the inner workings of hadoop/hbase IO have the time to help me confirm that this is the issue I am experiencing?
>
> 2010-07-15 16:33:26
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.2-b01 mixed mode):
>
> "RMI TCP Connection(73)-192.168.100.113" - Thread t@62011
>   java.lang.Thread.State: TIMED_WAITING on com.sun.jmx.remote.internal.ArrayNotificationBuffer@bd5192
>    at java.lang.Object.wait(Native Method)
>    at com.sun.jmx.remote.internal.ArrayNotificationBuffer.fetchNotifications(ArrayNotificationBuffer.java:417)
>    at com.sun.jmx.remote.internal.ArrayNotificationBuffer$ShareBuffer.fetchNotifications(ArrayNotificationBuffer.java:209)
>    at com.sun.jmx.remote.internal.ServerNotifForwarder.fetchNotifs(ServerNotifForwarder.java:258)
>    at javax.management.remote.rmi.RMIConnectionImpl$2.run(RMIConnectionImpl.java:1227)
>    at javax.management.remote.rmi.RMIConnectionImpl$2.run(RMIConnectionImpl.java:1225)
>    at javax.management.remote.rmi.RMIConnectionImpl.fetchNotifications(RMIConnectionImpl.java:1231)
>    at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>    at sun.rmi.transport.Transport$1.run(Transport.java:159)
>    at java.security.AccessController.doPrivileged(Native Method)
>    at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>    at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - locked java.util.concurrent.locks.ReentrantLock$NonfairSync@40aa3008
>
> "RMI TCP Connection(72)-192.168.100.113" - Thread t@62010
>   java.lang.Thread.State: RUNNABLE
>    at java.net.SocketInputStream.socketRead0(Native Method)
>    at java.net.SocketInputStream.read(SocketInputStream.java:129)
>    at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>    at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>    - locked java.io.BufferedInputStream@72bda36f
>    at java.io.FilterInputStream.read(FilterInputStream.java:66)
>    at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:517)
>    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - locked java.util.concurrent.locks.ReentrantLock$NonfairSync@350743
>
> "JMX server connection timeout 62009" - Thread t@62009
>   java.lang.Thread.State: TIMED_WAITING on [I@5eebd2c6
>    at java.lang.Object.wait(Native Method)
>    at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "ResponseProcessor for block blk_1926230463847049982_2694658" - Thread t@61160
>   java.lang.Thread.State: RUNNABLE
>    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
>    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>    - locked sun.nio.ch.Util$1@196fbfd0
>    - locked java.util.Collections$UnmodifiableSet@7799fdbb
>    - locked sun.nio.ch.EPollSelectorImpl@1ee13d55
>    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>    at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:332)
>    at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>    at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>    at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>    at java.io.DataInputStream.readFully(DataInputStream.java:178)
>    at java.io.DataInputStream.readLong(DataInputStream.java:399)
>    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:2399)
>
>   Locked ownable synchronizers:
>    - None
>
> "DataStreamer for file /hbase/.logs/dn01.colo.networkedinsights.com,60020,1279222293084/hlog.dat.1279228611023 block blk_1926230463847049982_2694658" - Thread t@61158
>   java.lang.Thread.State: TIMED_WAITING on java.util.LinkedList@475b455c
>    at java.lang.Object.wait(Native Method)
>    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2247)
>
>   Locked ownable synchronizers:
>    - None
>
> "RMI TCP Connection(71)-192.168.100.113" - Thread t@53904
>   java.lang.Thread.State: RUNNABLE
>    at sun.management.ThreadImpl.dumpThreads0(Native Method)
>    at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:374)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:167)
>    at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:96)
>    at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:33)
>    at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
>    at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
>    at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
>    at javax.management.StandardMBean.invoke(StandardMBean.java:391)
>    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>    at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>    at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
>    at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>    at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
>    at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)
>    at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>    at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>    at sun.rmi.transport.Transport$1.run(Transport.java:159)
>    at java.security.AccessController.doPrivileged(Native Method)
>    at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>    at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - locked java.util.concurrent.locks.ReentrantLock$NonfairSync@5581a749
>
> "JMX server connection timeout 28949" - Thread t@28949
>   java.lang.Thread.State: TIMED_WAITING on [I@1c6e538d
>    at java.lang.Object.wait(Native Method)
>    at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "RMI TCP Connection(70)-192.168.100.113" - Thread t@28947
>   java.lang.Thread.State: RUNNABLE
>    at java.net.SocketInputStream.socketRead0(Native Method)
>    at java.net.SocketInputStream.read(SocketInputStream.java:129)
>    at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>    at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>    - locked java.io.BufferedInputStream@513a56c5
>    at java.io.FilterInputStream.read(FilterInputStream.java:66)
>    at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:517)
>    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - locked java.util.concurrent.locks.ReentrantLock$NonfairSync@4d16c92c
>
> "regionserver/192.168.200.31:60020.compactor-EventThread" - Thread t@19245
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@35c7b446
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:414)
>
>   Locked ownable synchronizers:
>    - None
>
> "regionserver/192.168.200.31:60020.compactor-SendThread" - Thread t@19244
>   java.lang.Thread.State: RUNNABLE
>    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
>    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>    - locked sun.nio.ch.Util$1@30f33929
>    - locked java.util.Collections$UnmodifiableSet@ea26dec
>    - locked sun.nio.ch.EPollSelectorImpl@3905bf36
>    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:921)
>
>   Locked ownable synchronizers:
>    - None
>
> "RMI Scheduler(0)" - Thread t@406
>   java.lang.Thread.State: TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@66d9c57f
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
>    at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
>    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
>    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "pool-1-thread-1" - Thread t@74
>   java.lang.Thread.State: TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3175e1a2
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
>    at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
>    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
>    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "LruBlockCache.EvictionThread" - Thread t@72
>   java.lang.Thread.State: WAITING on org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread@32306c0c
>    at java.lang.Object.wait(Native Method)
>    at java.lang.Object.wait(Object.java:485)
>    at org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread.run(LruBlockCache.java:512)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 24 on 60020" - Thread t@71
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 23 on 60020" - Thread t@70
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 22 on 60020" - Thread t@69
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 21 on 60020" - Thread t@68
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 20 on 60020" - Thread t@67
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 19 on 60020" - Thread t@66
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 18 on 60020" - Thread t@65
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 17 on 60020" - Thread t@64
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 16 on 60020" - Thread t@63
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 15 on 60020" - Thread t@62
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 14 on 60020" - Thread t@61
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 13 on 60020" - Thread t@60
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 12 on 60020" - Thread t@59
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 11 on 60020" - Thread t@58
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 10 on 60020" - Thread t@57
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 9 on 60020" - Thread t@56
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 8 on 60020" - Thread t@55
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 7 on 60020" - Thread t@54
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 6 on 60020" - Thread t@53
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 5 on 60020" - Thread t@52
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 4 on 60020" - Thread t@51
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 3 on 60020" - Thread t@50
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 2 on 60020" - Thread t@49
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 1 on 60020" - Thread t@48
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server handler 0 on 60020" - Thread t@47
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@905a192
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server listener on 60020" - Thread t@14
>   java.lang.Thread.State: RUNNABLE
>    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
>    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>    - locked sun.nio.ch.Util$1@284c8f79
>    - locked java.util.Collections$UnmodifiableSet@3a5f659a
>    - locked sun.nio.ch.EPollSelectorImpl@4d077f66
>    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.run(HBaseServer.java:308)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Server Responder" - Thread t@16
>   java.lang.Thread.State: RUNNABLE
>    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
>    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>    - locked sun.nio.ch.Util$1@273d290c
>    - locked java.util.Collections$UnmodifiableSet@2051e6f4
>    - locked sun.nio.ch.EPollSelectorImpl@6111adbf
>    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>    at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.run(HBaseServer.java:477)
>
>   Locked ownable synchronizers:
>    - None
>
> "Timer-0" - Thread t@46
>   java.lang.Thread.State: TIMED_WAITING on java.util.TaskQueue@50b27cde
>    at java.lang.Object.wait(Native Method)
>    at java.util.TimerThread.mainLoop(Timer.java:509)
>    at java.util.TimerThread.run(Timer.java:462)
>
>   Locked ownable synchronizers:
>    - None
>
> "889074013@qtp0-1 - Acceptor0 SelectChannelConnector@0.0.0.0:60030" - Thread t@45
>   java.lang.Thread.State: RUNNABLE
>    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
>    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>    - locked sun.nio.ch.Util$1@1d7082c8
>    - locked java.util.Collections$UnmodifiableSet@b9bc44a
>    - locked sun.nio.ch.EPollSelectorImpl@6f7f73f7
>    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>    at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:429)
>    at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:185)
>    at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
>    at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:707)
>    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
>
>   Locked ownable synchronizers:
>    - None
>
> "1374821456@qtp0-0" - Thread t@44
>   java.lang.Thread.State: TIMED_WAITING on org.mortbay.thread.QueuedThreadPool$PoolThread@51f21c50
>    at java.lang.Object.wait(Native Method)
>    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:565)
>
>   Locked ownable synchronizers:
>    - None
>
> "regionserver/192.168.200.31:60020.leaseChecker" - Thread t@23
>   java.lang.Thread.State: TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7623fc35
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
>    at java.util.concurrent.DelayQueue.poll(DelayQueue.java:201)
>    at org.apache.hadoop.hbase.Leases.run(Leases.java:81)
>
>   Locked ownable synchronizers:
>    - None
>
> "regionserver/192.168.200.31:60020.majorCompactionChecker" - Thread t@22
>   java.lang.Thread.State: TIMED_WAITING
>    at java.lang.Thread.sleep(Native Method)
>    at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:74)
>    at org.apache.hadoop.hbase.Chore.run(Chore.java:76)
>
>   Locked ownable synchronizers:
>    - None
>
> "regionserver/192.168.200.31:60020.worker" - Thread t@17
>   java.lang.Thread.State: TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3483382b
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
>    at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:395)
>    at org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1425)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "regionserver/192.168.200.31:60020.compactor" - Thread t@19
>   java.lang.Thread.State: TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@12dca9fc
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
>    at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:395)
>    at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:85)
>
>   Locked ownable synchronizers:
>    - None
>
> "regionserver/192.168.200.31:60020.cacheFlusher" - Thread t@18
>   java.lang.Thread.State: TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@551be910
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
>    at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:395)
>    at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:145)
>
>   Locked ownable synchronizers:
>    - None
>
> "regionserver/192.168.200.31:60020.logFlusher" - Thread t@21
>   java.lang.Thread.State: TIMED_WAITING
>    at java.lang.Thread.sleep(Native Method)
>    at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:74)
>    at org.apache.hadoop.hbase.Chore.run(Chore.java:76)
>
>   Locked ownable synchronizers:
>    - None
>
> "regionserver/192.168.200.31:60020.logRoller" - Thread t@20
>   java.lang.Thread.State: TIMED_WAITING on java.util.concurrent.atomic.AtomicBoolean@1b624517
>    at java.lang.Object.wait(Native Method)
>    at org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:65)
>
>   Locked ownable synchronizers:
>    - None
>
> "Timer thread for monitoring jvm" - Thread t@42
>   java.lang.Thread.State: TIMED_WAITING on java.util.TaskQueue@63354999
>    at java.lang.Object.wait(Native Method)
>    at java.util.TimerThread.mainLoop(Timer.java:509)
>    at java.util.TimerThread.run(Timer.java:462)
>
>   Locked ownable synchronizers:
>    - None
>
> "Timer thread for monitoring hbase" - Thread t@41
>   java.lang.Thread.State: TIMED_WAITING on java.util.TaskQueue@4c966553
>    at java.lang.Object.wait(Native Method)
>    at java.util.TimerThread.mainLoop(Timer.java:509)
>    at java.util.TimerThread.run(Timer.java:462)
>
>   Locked ownable synchronizers:
>    - None
>
> "LeaseChecker" - Thread t@40
>   java.lang.Thread.State: TIMED_WAITING
>    at java.lang.Thread.sleep(Native Method)
>    at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.run(DFSClient.java:1066)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "IPC Client (47) connection to /192.168.200.30:60000 from an unknown user" - Thread t@28
>   java.lang.Thread.State: TIMED_WAITING on org.apache.hadoop.hbase.ipc.HBaseClient$Connection@3ffe5e31
>    at java.lang.Object.wait(Native Method)
>    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.waitForWork(HBaseClient.java:404)
>    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:447)
>
>   Locked ownable synchronizers:
>    - None
>
> "DestroyJavaVM" - Thread t@27
>   java.lang.Thread.State: RUNNABLE
>
>   Locked ownable synchronizers:
>    - None
>
> "regionserver/192.168.200.31:60020" - Thread t@26
>   java.lang.Thread.State: TIMED_WAITING
>    at java.lang.Thread.sleep(Native Method)
>    at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:74)
>    at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:594)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "main-EventThread" - Thread t@25
>   java.lang.Thread.State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54da03c9
>    at sun.misc.Unsafe.park(Native Method)
>    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:414)
>
>   Locked ownable synchronizers:
>    - None
>
> "main-SendThread" - Thread t@24
>   java.lang.Thread.State: RUNNABLE
>    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
>    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>    - locked sun.nio.ch.Util$1@18969651
>    - locked java.util.Collections$UnmodifiableSet@4666ac73
>    - locked sun.nio.ch.EPollSelectorImpl@7ec3d620
>    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:921)
>
>   Locked ownable synchronizers:
>    - None
>
> "Timer thread for monitoring rpc" - Thread t@15
>   java.lang.Thread.State: TIMED_WAITING on java.util.TaskQueue@352eaca4
>    at java.lang.Object.wait(Native Method)
>    at java.util.TimerThread.mainLoop(Timer.java:509)
>    at java.util.TimerThread.run(Timer.java:462)
>
>   Locked ownable synchronizers:
>    - None
>
> "RMI TCP Accept-0" - Thread t@12
>   java.lang.Thread.State: RUNNABLE
>    at java.net.PlainSocketImpl.socketAccept(Native Method)
>    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
>    - locked java.net.SocksSocketImpl@6b1f0fe4
>    at java.net.ServerSocket.implAccept(ServerSocket.java:453)
>    at java.net.ServerSocket.accept(ServerSocket.java:421)
>    at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:34)
>    at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
>    at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "RMI TCP Accept-10102" - Thread t@11
>   java.lang.Thread.State: RUNNABLE
>    at java.net.PlainSocketImpl.socketAccept(Native Method)
>    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
>    - locked java.net.SocksSocketImpl@444c78ad
>    at java.net.ServerSocket.implAccept(ServerSocket.java:453)
>    at java.net.ServerSocket.accept(ServerSocket.java:421)
>    at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
>    at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "RMI TCP Accept-0" - Thread t@10
>   java.lang.Thread.State: RUNNABLE
>    at java.net.PlainSocketImpl.socketAccept(Native Method)
>    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
>    - locked java.net.SocksSocketImpl@7886c8e2
>    at java.net.ServerSocket.implAccept(ServerSocket.java:453)
>    at java.net.ServerSocket.accept(ServerSocket.java:421)
>    at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
>    at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>    at java.lang.Thread.run(Thread.java:619)
>
>   Locked ownable synchronizers:
>    - None
>
> "Signal Dispatcher" - Thread t@5
>   java.lang.Thread.State: RUNNABLE
>
>   Locked ownable synchronizers:
>    - None
>
> "Finalizer" - Thread t@3
>   java.lang.Thread.State: WAITING on java.lang.ref.ReferenceQueue$Lock@b485e43
>    at java.lang.Object.wait(Native Method)
>    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
>    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
>    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>
>   Locked ownable synchronizers:
>    - None
>
> "Reference Handler" - Thread t@2
>   java.lang.Thread.State: WAITING on java.lang.ref.Reference$Lock@3c24867e
>    at java.lang.Object.wait(Native Method)
>    at java.lang.Object.wait(Object.java:485)
>    at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
>
>   Locked ownable synchronizers:
>    - None
>
> ---
>
> Respectfully Yours,
> Luke Forehand
> Software Engineer
> Networked Insights
> http://www.networkedinsights.com
>



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message