hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Gonzalez <Robert.Gonza...@maxpointinteractive.com>
Subject RE: wrong region exception
Date Tue, 31 May 2011 17:42:29 GMT
Now I'm getting the wrong region exception on the new table that I'm copying the old table
to.  Running hbck reveals an inconsistency in the new table.  The frustration is unbelievable.
 Like I said before, it doesn't appear that HBase is ready for prime time.  I don't know how
companies are using this successfully, it doesn't appear plausible.


-----Original Message-----
From: Robert Gonzalez [mailto:Robert.Gonzalez@maxpointinteractive.com] 
Sent: Tuesday, May 31, 2011 11:03 AM
To: 'user@hbase.apache.org'
Subject: RE: wrong region exception

I'm trying my "nuclear" option: basically copy the data from the old db to a new one, skipping
over bad regions.  The bad news is that it is taking forever.

I get a stack trace just trying to run check_meta.rb:

maxpoint@c1-m02:/usr/lib/hbase/bin$ ./hbase org.jruby.Main check_meta.rb Writables.java:75:in
`org.apache.hadoop.hbase.util.Writables.getWritable': java.lang.NullPointerException: null
	from Writables.java:119:in `org.apache.hadoop.hbase.util.Writables.getHRegionInfo'
	from NativeMethodAccessorImpl.java:-2:in `sun.reflect.NativeMethodAccessorImpl.invoke0'
	from NativeMethodAccessorImpl.java:39:in `sun.reflect.NativeMethodAccessorImpl.invoke'
	from DelegatingMethodAccessorImpl.java:25:in `sun.reflect.DelegatingMethodAccessorImpl.invoke'
	from Method.java:597:in `java.lang.reflect.Method.invoke'
	from JavaMethod.java:196:in `org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling'
	from JavaMethod.java:182:in `org.jruby.javasupport.JavaMethod.invoke_static'
	from JavaClass.java:371:in `org.jruby.javasupport.JavaClass$StaticMethodInvoker.execute'
	 ... 17 levels...
	from Main.java:183:in `org.jruby.Main.runInterpreter'
	from Main.java:120:in `org.jruby.Main.run'
	from Main.java:95:in `org.jruby.Main.main'
Complete Java stackTrace
	at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75)
	at org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:119)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling(JavaMethod.java:196)
	at org.jruby.javasupport.JavaMethod.invoke_static(JavaMethod.java:182)
	at org.jruby.javasupport.JavaClass$StaticMethodInvoker.execute(JavaClass.java:371)
	at org.jruby.internal.runtime.methods.SimpleCallbackMethod.call(SimpleCallbackMethod.java:81)
	at org.jruby.evaluator.EvaluationState.callNode(EvaluationState.java:571)
	at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:207)
	at org.jruby.evaluator.EvaluationState.localAsgnNode(EvaluationState.java:1254)
	at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:286)
	at org.jruby.evaluator.EvaluationState.blockNode(EvaluationState.java:533)
	at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:201)
	at org.jruby.evaluator.EvaluationState.whileNode(EvaluationState.java:1793)
	at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:387)
	at org.jruby.evaluator.EvaluationState.blockNode(EvaluationState.java:533)
	at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:201)
	at org.jruby.evaluator.EvaluationState.rootNode(EvaluationState.java:1628)
	at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:356)
	at org.jruby.evaluator.EvaluationState.eval(EvaluationState.java:164)
	at org.jruby.Ruby.eval(Ruby.java:278)
	at org.jruby.Ruby.compileOrFallbackAndRun(Ruby.java:306)
	at org.jruby.Main.runInterpreter(Main.java:238)
	at org.jruby.Main.runInterpreter(Main.java:183)
	at org.jruby.Main.run(Main.java:120)
	at org.jruby.Main.main(Main.java:95)

-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Friday, May 27, 2011 12:43 AM
To: user@hbase.apache.org
Subject: Re: wrong region exception


Looks like script already exists.  Check bin/check_meta.rb.  If you pass it --fix it should
plug the hole.  Read the head of the script for how to run it.

Good luck,

On Thu, May 26, 2011 at 1:06 PM, Robert Gonzalez <Robert.Gonzalez@maxpointinteractive.com>
> I sent the meta.txt to your saint.ack@gmail .com account due to the attachment.
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of 
> Stack
> Sent: Thursday, May 26, 2011 1:35 PM
> To: user@hbase.apache.org
> Subject: Re: wrong region exception
> On Thu, May 26, 2011 at 8:06 AM, Robert Gonzalez <Robert.Gonzalez@maxpointinteractive.com>
>> It looks like an entire region is missing, here is the online table:
>> urlhashv2,7FF1A5BF839C37078083B4F8267008F6,1303028235302.b0d55566fd0e
>> 0 2ae98541618396aa7b1.
>> c1-s03.atxd.maxpointinteractive.com:60030
>> 7FF1A5BF839C37078083B4F8267008F6
>> 80116D7E506D87ED39EAFFE784B5B590
>> urlhashv2,8031483E0B3B7F587020FCBB764272D9,1305226123483.3ed065ad87f8
>> 9 aece6b994dd31b42b2a.      c1-s33.atxd.maxpointinteractive.com:60030
>> 8031483E0B3B7F587020FCBB764272D9
>> 8041346D0B05617FA4B9152BFE9B18B9
>> One ends at 80116D7E506D87ED39EAFFE784B5B590, but the next one doesn't start there.
> So, make sure you actually have a hole.  Dump out your meta table:
> echo "scan '.META.'"| ./bin/hbase shell &> /tmp/meta.txt
> Then look ensure that there is a hole between the above regions (compare start and end
keys... the end key of one region needs to match the start key of the next).
> If indeed a hole, you need to do a little surgery inserting a new missing region (hbck
should fix this but it doesn't have the smarts just yet).
> Basically, you create a new region with start and end keys to fill the hole then you
insert it into .META. and then assign it.  There are some scripts in our bin directory that
do various parts of this.  I'm pretty sure its beyond any but a few figuring this mess out
so if you do the above foot work and provide a few more details, I'll hack up something for
you (and hopefully something generalized to be use by others later, and later to be integrated
into hbck).
> St.Ack

View raw message