hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amanda Moran <amanda.mo...@esgyn.com>
Subject Re: Issues with HBase Regions in Transition
Date Wed, 02 Dec 2015 20:51:58 GMT
Hi there Ted-

Thanks for response. Update: I copied my trafodion.jar files back to
../hbase/lib , did a restart and then hbase is back online... I then
removed them again (just to make sure) did a restart .. and then its back
in the region in transition. Seems like we should be able to remove our
files and do a restart and have hbase be back online... Is this incorrect?

Log snippet:

2015-12-02 19:48:41,614 ERROR
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
of region=hbase:meta,,1.1588230740, starting to roll back the global
memstore size.
java.lang.IllegalStateException: Could not instantiate a region instance.
        at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:5486)
        at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5793)
        at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5765)
        at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5721)
        at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5672)
        at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:356)
        at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:126)
        at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
Class
org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion not
found
        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2112)
        at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:5475)
        ... 10 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion not
found
        at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2018)
        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2110)
        ... 11 more
2015-12-02 19:48:41,627 INFO
org.apache.hadoop.hbase.coordination.ZkOpenRegionCoordination: Opening of
region {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '',
ENDKEY => ''} failed, transitioning from OPENING to FAILED_OPEN in ZK,
expecting version 4
2015-12-02 19:48:41,673 INFO
org.apache.hadoop.hbase.regionserver.RSRpcServices: Open
hbase:meta,,1.1588230740
2015-12-02 19:48:41,679 ERROR
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
of region=hbase:meta,,1.1588230740, starting to roll back the global
memstore size.
java.lang.IllegalStateException: Could not instantiate a region instance.
        at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:5486)
        at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5793)
        at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5765)
        at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5721)
        at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5672)
        at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:356)
        at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:126)
        at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
Class
org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion not
found
        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2112)
        at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:5475)
        ... 10 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion not
found
        at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2018)
        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2110)
        ... 11 more


Let me know if you need more from the logs.

THANKS!

On Wed, Dec 2, 2015 at 12:37 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Do you mind pastebin snippet of region server log where the region stuck in
> transition was hosted ?
>
> This would give us some clue.
>
> Cheers
>
> On Wed, Dec 2, 2015 at 12:14 PM, Amanda Moran <amanda.moran@esgyn.com>
> wrote:
>
> > Hi there All-
> >
> > I apologize if this issue has been raised before... I have done ALOT of
> > googling on this issues... and finally decided to just write here to the
> > source :)
> >
> > Background:
> >
> > I work on the incubating Apache Trafodion project.
> >
> > I am currently using HBase 1.0 with Cloudera 5.4.5.
> >
> > I have tried many times to upgrade my cloudera cluster (to get the latest
> > version of HBase) and have always failed due to my HBase master failing
> to
> > come backup do to a Region suck in Transition. It is always the
> hbase:meta.
> >
> > In the past I just assumed that the upgrade feature just didn't actually
> > work... and so even though frustrating... I just let it go.
> >
> > But now I have come across this issue again, in a slightly different
> way...
> > and this makes me think it is an HBase problem.
> >
> > I am preparing for an upgrade, and because of this I decided to remove
> the
> > "Trafodion".jar files that are located in the ../hbase/lib directory. I
> > thought this might be the cause of not being able to upgrade, because I
> was
> > leaving the trafodion files in this directory. I removed them, then did a
> > deploy client config (since some configs were stale), and then did a
> > restart. Now I have a region stuck in transition, even before the
> upgrade!
> >
> > Everything I have seen on the web, has not worked in the past. How do I
> get
> > pass this issue? Do I have no choice but to do a full uninstall/reinstall
> > and lose my data?
> >
> > Thanks all for any help!
> >
> >
> > --
> > Thanks,
> >
> > Amanda Moran
> >
>



-- 
Thanks,

Amanda Moran

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message