hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: checkAndPut() failing with NotServingRegionException
Date Thu, 23 Jun 2011 17:20:20 GMT
I like this one:
>> like making a malnourished child work in a coal mine

On Thu, Jun 23, 2011 at 10:03 AM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Getting RetriesExhaustedWithDetailsException due to NSRE means that it
> took forever for a region server to close or split a region, what you
> pasted from the region server talks a region closing but that also
> happens during split.
>
> I'd suggest digging more in those region server logs using this guide:
> http://hbase.apache.org/book/trouble.html
>
> Also make sure you review this
> http://hbase.apache.org/book/performance.html
>
> Finally giving a 1GB heap to HBase while inserting a lot of data is
> like making a malnourished child work in a coal mine, it's not very
> nice of you.
>
> J-D
>
> On Wed, Jun 22, 2011 at 11:06 PM, Sam Seigal <selekt86@yahoo.com> wrote:
> > Hi,
> >
> > I am loading data into my HBase cluster and running into two issues -
> >
> > During my import, I received the following exception ->
> >
> > org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> Failed
> > 53484 actions: servers with issues: spock7001:60020,
> >        at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1220)
> >        at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1234)
> >        at
> > org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> >        at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:675)
> >        at org.apache.hadoop.hbase.client.HTable.put(HTable.java:660)
> >
> > May have cluster issues => true
> > Cause 0
> >
> > When I check the logs on the regions server, the last thrown exception is
> > the following =>
> >
> > Thu Jun 23 05:16:18 2011 GMT regionserver 10460-0@spock7001:0 [DEBUG]
> (IPC
> > Server handler 7 on 60020)
> > { org.apache.hadoop.hbase.NotServingRegionException:
> > hbaseTable,,1308805558566.5aefc6c2b9599f55f8b40351a61db03c. is closing
> > Thu Jun 23 05:22:18 2011 GMT regionserver 10460-0@spock7001:0 [DEBUG]
> > (regionserver60020.logRoller) org.apache.hadoop.conf.Configuration:
> > java.io.IOException: config()
> >
> > On running status 'detailed' in the shell , I get =>
> >
> > 0 regionsInTransition
> > 3 live servers
> >   spock7001:60020 1308805454136
> >        requests=0, regions=0, usedHeap=470, maxHeap=910
> >    spock6002:60020 1308805434201
> >        requests=0, regions=1, usedHeap=550, maxHeap=910
> >        hbaseTable,,1308805558566.5aefc6c2b9599f55f8b40351a61db03c.
> >            stores=1, storefiles=2, storefileSizeMB=383, memstoreSizeMB=0,
> > storefileIndexSizeMB=1
> >    spock6001:60020 1308805268507
> >        requests=0, regions=2, usedHeap=90, maxHeap=910
> >        -ROOT-,,0
> >            stores=1, storefiles=1, storefileSizeMB=0, memstoreSizeMB=0,
> > storefileIndexSizeMB=0
> >        .META.,,1
> >            stores=1, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > storefileIndexSizeMB=0
> > 0 dead servers
> >
> >
> > I am issuing a checkAndPut() to insert records into HBase. Is this a bug
> ?
> >
> > Secondly, I have followed the instructions in the HBase book to increase
> > write throughput. I have the following settings for my hbase table:
> >
> > config = HBaseConfiguration.create();
> > table = new HTable (config, "hbaseTable");
> > table.setAutoFlush(false);
> > table.setWriteBufferSize(104857600);
> >
> > However, according to my logs, each checkAndPut() call takes on an
> average
> > of 5 milliseconds. Is this unavoidable overhead due to locking ?
> >
> > All of my HBase daemons are running with -Xmx1g of heapsize.
> >
> > Any help is appreciated.
> >
> > Thank you,
> >
> > Sam
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message