I ran some benchmarks comparing the two and actual performance is the
same, the perception came from HBASE 18469 [1] where the request/s
numbers were changed for 1.4.0 and 2.0.0.
Thanks,
Austin
[1] https://issues.apache.org/jira/browse/HBASE-18469
On 07/26/2018 06:41 PM, Stack wrote:
> On Mon, Jul 23, 2018 at 6:57 AM Austin Heyne <aheyne@ccri.com> wrote:
>
>> Hey all, hopefully this is a simple one.
>>
>> We've migrated from HBase (on s3) 1.3.2 to 1.4.4, creating a fresh
>> instance with all our configuration from the previous one with few
>> changes. However, we're seeing ingest run ~100x slower than we saw on
>> 1.3.2 and the requests per second seem really low at around 3k total for
>> a 10 node cluster. We're not seeing any atypical errors but it does seem
>> we're seeing a ton of flushes, 1 about every 3 or 4 seconds but the logs
>> indicate the flushes are full size ~128MB. I've included our
>> configuration below.
>>
>>
> Check old logs to see what rate you used to flush at?
>
> Is it possible that you are just writing way more data now or the data
> character is different now? Larger values?
>
> S
>
>
>
>> Thanks for the help,
>> Austin
>>
>> [
>> {
>> "classification": "hbase-site",
>> "properties": {
>> "fs.s3.consistent.retryPeriodSeconds": "10",
>> "hbase.regionserver.thread.compaction.large": "3",
>> "fs.s3.consistent.retryPolicyType": "fixed",
>> "hbase.hstore.blockingStoreFiles": "1000",
>> "fs.s3.consistent.throwExceptionOnInconsistency": "false",
>> "hbase.bucketcache.size": "27000",
>> "hbase.ipc.server.callqueue.read.ratio": "0.25",
>> "hbase.bucketcache.combinedcache.enabled": "true",
>> "fs.s3a.threads.max": "50",
>> "hbase.regionserver.thread.compaction.small": "2",
>> "hbase.hregion.memstore.flush.size": "134217728",
>> "hbase.hregion.max.filesize": "21474836480",
>> "hbase.regionserver.regionSplitLimit": "10000",
>> "fs.s3.consistent.metadata.tableName": "redacted",
>> "hbase.hstore.compaction.max": "1000",
>> "hbase.regionserver.global.memstore.size": "0.4",
>> "hbase.ipc.server.callqueue.handler.factor": "0.5",
>> "hbase.regionserver.logroll.period": "100000",
>> "hbase.hregion.majorcompaction": "0",
>> "hbase.hstore.compactionThreshold": "1000",
>> "hbase.hregion.memstore.mslab.enabled": "false",
>> "hbase.regionserver.handler.count": "50",
>> "fs.s3a.connection.maximum": "100",
>> "hbase.hstore.flusher.count": "10",
>> "hbase.hstore.blockingWaitTime": "0",
>> "hbase.hregion.memstore.block.multiplier": "10",
>> "hbase.bucketcache.ioengine": "offheap"
>> }
>> },
>> {
>> "configurations": [
>> {
>> "classification": "export",
>> "properties": {
>> "HBASE_REGIONSERVER_OPTS": "\"-Dcom.sun.management.jmxremote
>> -Dcom.sun.management.jmxremote.authenticate\u003dfalse
>> -Dcom.sun.management.jmxremote.port\u003d10102
>> -Dcom.sun.management.jmxremote.ssl\u003dfalse -Xmx28G
>> -XX:MaxDirectMemorySize\u003d28G\"",
>> "HBASE_MASTER_OPTS": "\"-Dcom.sun.management.jmxremote
>> -Dcom.sun.management.jmxremote.authenticate\u003dfalse
>> -Dcom.sun.management.jmxremote.port\u003d10101
>> -Dcom.sun.management.jmxremote.ssl\u003dfalse -Xmx28G
>> -XX:MaxDirectMemorySize\u003d28G\""
>> }
>> }
>> ],
>> "classification": "hbase-env",
>> "properties": {
>>
>> }
>> },
>> {
>> "classification": "hbase-metrics",
>> "properties": {
>> "rpc.period": "60",
>> "hbase.period": "60",
>> "rpc.class":
>> "org.apache.hadoop.metrics.spi.NullContextWithUpdateThread",
>> "hbase.class":
>> "org.apache.hadoop.metrics.spi.NullContextWithUpdateThread",
>> "jvm.class":
>> "org.apache.hadoop.metrics.spi.NullContextWithUpdateThread",
>> "jvm.period": "60"
>> }
>> },
>> {
>> "classification": "emrfs-site",
>> "properties": {
>> "fs.s3.consistent": "true"
>> }
>> }
>> ]
>>
>> --
>> Austin L. Heyne
>>
>>
--
Austin L. Heyne
|