hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Alves <dr-al...@criticalsoftware.com>
Subject Lost Rows
Date Wed, 23 Apr 2008 13:56:02 GMT
Hi Guys

	Regarding my previous problems  I'm glad to say that I can now crawl an
entire repository with only a small percentage of failed tasks, last
hbase version plus the correction of replication property seemed to
solve it for me.
	Still I have two issues I'd appreciate your input in. 
	The first one regards splits. I've made a small tool (built upon
stack's one) that checks DB state, and can online/offline tables and
merge regions etc. This tool gives me the report ant the end of this
email. The question here Is that I seem to have lost 144 rows (comparing
the output formats output records and the actual rows in the table from
a select count(*)). I suspect these rows are in the offline splits. Can
I use my tool to merge the splits against their online parents using
HRegion.merge() ? Or is it a big no no.
	The second issue is more problematic, I misconfigured my last job and
it ran 10 maps instead of the 1 it should, but when under that kind of
load hbase completely failed, regionservers went down, at one time I had
to completely erase the database because it wouldn't start again (I
suspect .META. was offline) the other time I was able to recover all the
data by simply restarting it. Is there any kind of procedure I should
use in this situation?

Best Regards
David Alves

Log Trace:
Found region: cyclops-documents-database,,1208892792201
Id: 1208892792201
Start Key: 
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/HOW
TO USE HTML 3.2/ch6.htm
Online/Offline Status: ONLINE
Split?: FALSE
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/HOW
TO USE HTML 3.2/ch6.htm,1208892792202
Id: 1208892792202
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/HOW
TO USE HTML 3.2/ch6.htm
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/LINUX
SYSTEM ADMINISTRATOR'S SURVIVAL GUIDE TABLE OF CONTENTS/lsg14.htm
Online/Offline Status: ONLINE
Split?: FALSE
DEBUG 23-04 14:54:50,744 (DFSClient.java:readChunk:934)  -DFSClient
readChunk got seqno 2 offsetInBlock 8192 lastPacketInBlock false
packetLen 4132
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/LINUX
SYSTEM ADMINISTRATOR'S SURVIVAL GUIDE TABLE OF CONTENTS/lsg14.htm,1208891918491
Id: 1208891918491
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/LINUX
SYSTEM ADMINISTRATOR'S SURVIVAL GUIDE TABLE OF CONTENTS/lsg14.htm
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/SPECIAL
EDITION USING MICROSOFT BACKOFFICE, VOLUME 1/ch05/06.htm
Online/Offline Status: OFFLINE
Split?: TRUE
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/LINUX
SYSTEM ADMINISTRATOR'S SURVIVAL GUIDE TABLE OF CONTENTS/lsg14.htm,1208893494772
Id: 1208893494772
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/LINUX
SYSTEM ADMINISTRATOR'S SURVIVAL GUIDE TABLE OF CONTENTS/lsg14.htm
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Platinium
Edition Using VB 5/Books/Platinium Edition Using VB 5/ch14/09.htm
Online/Offline Status: ONLINE
Split?: FALSE
DEBUG 23-04 14:54:50,754 (DFSClient.java:readChunk:934)  -DFSClient
readChunk got seqno 3 offsetInBlock 12288 lastPacketInBlock false
packetLen 4132
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Platinium
Edition Using VB 5/Books/Platinium Edition Using VB 5/ch14/09.htm,1208893494773
Id: 1208893494773
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Platinium
Edition Using VB 5/Books/Platinium Edition Using VB 5/ch14/09.htm
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/SPECIAL
EDITION USING MICROSOFT BACKOFFICE, VOLUME 1/ch05/06.htm
Online/Offline Status: OFFLINE
Split?: TRUE
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Platinium
Edition Using VB 5/Books/Platinium Edition Using VB 5/ch14/09.htm,1208894034845
Id: 1208894034845
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Platinium
Edition Using VB 5/Books/Platinium Edition Using VB 5/ch14/09.htm
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Platinium
Edition Using VB 5/Books/Platinium Edition Using VB 5/ch40/01.htm
Online/Offline Status: ONLINE
Split?: FALSE
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Platinium
Edition Using VB 5/Books/Platinium Edition Using VB 5/ch40/01.htm,1208896414707
Id: 1208896414707
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Platinium
Edition Using VB 5/Books/Platinium Edition Using VB 5/ch40/01.htm
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Programming/Delphi/Delphi
Informant [1995-2003]/Works/95index.PDF
Online/Offline Status: ONLINE
Split?: FALSE
DEBUG 23-04 14:54:50,756 (DFSClient.java:readChunk:934)  -DFSClient
readChunk got seqno 4 offsetInBlock 16384 lastPacketInBlock true
packetLen 3402
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Programming/Delphi/Delphi
Informant [1995-2003]/Works/95index.PDF,1208896478277
Id: 1208896478277
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Programming/Delphi/Delphi
Informant [1995-2003]/Works/95index.PDF
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Programming/Java/java-look-feel-design-guidelines-2nd/HIG.Text3.html
Online/Offline Status: ONLINE
Split?: FALSE
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Programming/Java/java-look-feel-design-guidelines-2nd/HIG.Text3.html,1208896478277
Id: 1208896478277
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/Programming/Java/java-look-feel-design-guidelines-2nd/HIG.Text3.html
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/SPECIAL
EDITION USING MICROSOFT BACKOFFICE, VOLUME 1/ch05/06.htm
Online/Offline Status: ONLINE
Split?: FALSE
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/SPECIAL
EDITION USING MICROSOFT BACKOFFICE, VOLUME 1/ch05/06.htm,1208891918491
Id: 1208891918491
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/SPECIAL
EDITION USING MICROSOFT BACKOFFICE, VOLUME 1/ch05/06.htm
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/SPECIAL
EDITION, USING LOTUS NOTES/e-book/ch25.htm
Online/Offline Status: ONLINE
Split?: FALSE
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/SPECIAL
EDITION, USING LOTUS NOTES/e-book/ch25.htm,1208891541773
Id: 1208891541773
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/SPECIAL
EDITION, USING LOTUS NOTES/e-book/ch25.htm
End Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/oreilly-cgionwww/oreilly-cgionwww/ch01_02.txt
Online/Offline Status: ONLINE
Split?: FALSE
Found region:
cyclops-documents-database,smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/oreilly-cgionwww/oreilly-cgionwww/ch01_02.txt,1208891541774
Id: 1208891541774
Start Key:
smb://cbrfileserv.critical.pt/CyclopsRepoLocation-/Critical/Biblioteca-CyclopsRepoLocation/EBooks/oreilly-cgionwww/oreilly-cgionwww/ch01_02.txt
End Key: 
Online/Offline Status: ONLINE
Split?: FALSE
Found region: cyclops-links-database,,1208891170959
Id: 1208891170959
Start Key: 
End Key: 
Online/Offline Status: ONLINE
Split?: FALSE
	
	  


Mime
View raw message