hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Goel, Ankur" <Ankur.G...@corp.aol.com>
Subject RE: Concurrent Modification Exception
Date Wed, 12 Mar 2008 06:35:35 GMT
Hi,
   I compile Hadoop and Hbase locally on my machine, so 
Hadoop version is '0.17.0-dev' and Hbase version is '0.2.0-dev'.

Also note that my machine is a windows XP machine and I use cygwin
for both compiling and running Hadoop/Hbase on my machine.

The stack trace from Region server logs is given below.

008-03-11 15:49:17,652 INFO
org.apache.hadoop.hbase.regionserver.HRegion: region
TestTable,0000097920,1205226364049 available
2008-03-11 15:49:17,652 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN :
TestTable,0029434692,1205229579914
2008-03-11 15:49:17,683 INFO
org.apache.hadoop.hbase.regionserver.HRegion: region
TestTable,0029434692,1205229579914 available
2008-03-11 15:49:17,683 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN :
TestTable,0006355562,1205229162173
2008-03-11 15:49:17,730 INFO
org.apache.hadoop.hbase.regionserver.HRegion: region
TestTable,0006355562,1205229162173 available
2008-03-11 15:49:17,730 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN :
TestTable,,1205226328794
2008-03-11 15:49:17,777 INFO
org.apache.hadoop.hbase.regionserver.HRegion: region
TestTable,,1205226328794 available
2008-03-11 15:55:38,395 WARN
org.apache.hadoop.hbase.regionserver.HRegionServer: Processing message
(Retry: 0)
java.io.IOException: java.io.IOException:
java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.j
ava:366)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell
(ServerManager.java:268)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerMa
nager.java:187)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:5
53)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:910)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
	at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
ccessorImpl.java:39)
	at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
tructorAccessorImpl.java:27)
	at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(Rem
oteExceptionHandler.java:82)
	at
org.apache.hadoop.hbase.RemoteExceptionHandler.checkIOException(RemoteEx
ceptionHandler.java:48)
	at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.jav
a:386)
	at java.lang.Thread.run(Thread.java:619)
2008-03-11 15:55:38,395 WARN
org.apache.hadoop.hbase.regionserver.HRegionServer: Processing message
(Retry: 1)
java.io.IOException: java.io.IOException:
java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.j
ava:366)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell
(ServerManager.java:268)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerMa
nager.java:187)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:5
53)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:910)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
	at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
ccessorImpl.java:39)
	at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
tructorAccessorImpl.java:27)
	at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(Rem
oteExceptionHandler.java:82)
	at
org.apache.hadoop.hbase.RemoteExceptionHandler.checkIOException(RemoteEx
ceptionHandler.java:48)
	at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.jav
a:386)
	at java.lang.Thread.run(Thread.java:619)
2008-03-11 15:55:38,395 WARN
org.apache.hadoop.hbase.regionserver.HRegionServer: Processing message
(Retry: 2)
java.io.IOException: java.io.IOException:
java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.j
ava:366)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell
(ServerManager.java:268)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerMa
nager.java:187)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:5
53)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:910)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
	at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
ccessorImpl.java:39)
	at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
tructorAccessorImpl.java:27)
	at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(Rem
oteExceptionHandler.java:82)
	at
org.apache.hadoop.hbase.RemoteExceptionHandler.checkIOException(RemoteEx
ceptionHandler.java:48)
	at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.jav
a:386)
	at java.lang.Thread.run(Thread.java:619)


Here are the logs from Hmaster

2008-03-11 15:46:19,591 INFO
org.apache.hadoop.hbase.master.ServerManager: 10.146.165.80:60020
serving TestTable,0018969406,1205229954640
2008-03-11 15:46:19,591 INFO
org.apache.hadoop.hbase.master.ServerManager: 10.146.165.80:60020
serving TestTable,0023087066,1205229579914
2008-03-11 15:46:19,591 INFO
org.apache.hadoop.hbase.master.ServerManager: 10.146.165.80:60020
serving TestTable,0000097920,1205226364049
2008-03-11 15:46:19,591 INFO
org.apache.hadoop.hbase.master.ServerManager: 10.146.165.80:60020
serving TestTable,0029434692,1205229579914
2008-03-11 15:46:19,591 INFO
org.apache.hadoop.hbase.master.ServerManager: 10.146.165.80:60020
serving TestTable,0006355562,1205229162173
2008-03-11 15:46:19,591 INFO
org.apache.hadoop.hbase.master.ServerManager: 10.146.165.80:60020
serving TestTable,,1205226328794
2008-03-11 15:46:19,591 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0002123598,1205226744066, startKey: <0002123598>, endKey:
<0004206876>, encodedName: 309209042, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,591 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0002123598,1205226744066 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,591 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0005281155,1205229224041, startKey: <0005281155>, endKey:
<0006355562>, encodedName: 1633815830, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0005281155,1205229224041 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0009448095,1205229850239, startKey: <0009448095>, endKey:
<0012621780>, encodedName: 1014866630, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0009448095,1205229850239 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0012621780,1205229850239, startKey: <0012621780>, endKey:
<0015795593>, encodedName: 165593217, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0012621780,1205229850239 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0007429841,1205229162173, startKey: <0007429841>, endKey:
<0009448095>, encodedName: 1851690796, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0007429841,1205229162173 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0015795593,1205229954640, startKey: <0015795593>, endKey:
<0018969406>, encodedName: 477232303, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0015795593,1205229954640 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0004206876,1205229224041, startKey: <0004206876>, endKey:
<0005281155>, encodedName: 247834599, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0004206876,1205229224041 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0018969406,1205229954640, startKey: <0018969406>, endKey:
<0023087066>, encodedName: 1686119456, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0018969406,1205229954640 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0023087066,1205229579914, startKey: <0023087066>, endKey:
<0029434692>, encodedName: 1844401047, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0023087066,1205229579914 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0000097920,1205226364049, startKey: <0000097920>, endKey:
<0002123598>, encodedName: 57245683, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0000097920,1205226364049 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0029434692,1205229579914, startKey: <0029434692>, endKey: <>,
encodedName: 1168116125, tableDesc: {name: TestTable, families:
{info:={name: info, max versions: 3, compression: NONE, in memory:
false, block cache enabled: false, max length: 2147483647, bloom filter:
none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0029434692,1205229579914 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,0006355562,1205229162173, startKey: <0006355562>, endKey:
<0007429841>, encodedName: 1182565503, tableDesc: {name: TestTable,
families: {info:={name: info, max versions: 3, compression: NONE, in
memory: false, block cache enabled: false, max length: 2147483647, bloom
filter: none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,606 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,0006355562,1205229162173 in table .META.,,1 with startcode
1205230563513 and server 10.146.165.80:60020
2008-03-11 15:46:19,622 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: regionname:
TestTable,,1205226328794, startKey: <>, endKey: <0000097920>,
encodedName: 2134019504, tableDesc: {name: TestTable, families:
{info:={name: info, max versions: 3, compression: NONE, in memory:
false, block cache enabled: false, max length: 2147483647, bloom filter:
none}}} open on 10.146.165.80:60020
2008-03-11 15:46:19,622 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: updating row
TestTable,,1205226328794 in table .META.,,1 with startcode 1205230563513
and server 10.146.165.80:60020
2008-03-11 15:47:01,325 INFO org.apache.hadoop.hbase.master.HMaster:
deleted table: TestTable
2008-03-11 15:47:01,638 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 60000, call regionServerReport(address:
10.146.165.80:60020, startcode: 1205230563513, load: (requests: 21
regions: 16), [Lorg.apache.hadoop.hbase.HMsg;@7e9ce2) from
10.146.165.80:2099: error: java.io.IOException:
java.util.ConcurrentModificationException
java.io.IOException: java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.j
ava:366)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell
(ServerManager.java:268)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerMa
nager.java:187)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:5
53)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:910)
2008-03-11 15:47:01,653 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 60000, call regionServerReport(address:
10.146.165.80:60020, startcode: 1205230563513, load: (requests: 0
regions: 16), [Lorg.apache.hadoop.hbase.HMsg;@39c8c1) from
10.146.165.80:2099: error: java.io.IOException:
java.util.ConcurrentModificationException
java.io.IOException: java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.j
ava:366)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell
(ServerManager.java:268)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerMa
nager.java:187)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:5
53)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:910)
2008-03-11 15:47:01,653 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 60000, call regionServerReport(address:
10.146.165.80:60020, startcode: 1205230563513, load: (requests: 0
regions: 16), [Lorg.apache.hadoop.hbase.HMsg;@111ded2) from
10.146.165.80:2099: error: java.io.IOException:
java.util.ConcurrentModificationException
java.io.IOException: java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.j
ava:366)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell
(ServerManager.java:268)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerMa
nager.java:187)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:5
53)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:910)
2008-03-11 15:47:01,653 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 60000, call regionServerReport(address:
10.146.165.80:60020, startcode: 1205230563513, load: (requests: 0
regions: 16), [Lorg.apache.hadoop.hbase.HMsg;@3bc20e) from
10.146.165.80:2099: error: java.io.IOException:
java.util.ConcurrentModificationException
java.io.IOException: java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.j
ava:366)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell
(ServerManager.java:268)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerMa
nager.java:187)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:5
53)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:910)
2008-03-11 15:47:01,653 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 60000, call regionServerReport(address:
10.146.165.80:60020, startcode: 1205230563513, load: (requests: 0
regions: 16), [Lorg.apache.hadoop.hbase.HMsg;@1415056) from
10.146.165.80:2099: error: java.io.IOException:
java.util.ConcurrentModificationException
java.io.IOException: java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.j
ava:366)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell
(ServerManager.java:268)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerMa
nager.java:187)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:5
53)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:910)
2008-03-11 15:47:01,653 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 60000, call regionServerReport(address:
10.146.165.80:60020, startcode: 1205230563513, load: (requests: 0
regions: 16), [Lorg.apache.hadoop.hbase.HMsg;@1014e21) from
10.146.165.80:2099: error: java.io.IOException:
java.util.ConcurrentModificationException
java.io.IOException: java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.j
ava:366)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell
(ServerManager.java:268)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerMa
nager.java:187)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:5
53)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:910)
2008-03-11 15:47:03,763 INFO
org.apache.hadoop.hbase.master.ServerManager: Cancelling lease for
10.146.165.80:60020
2008-03-11 15:47:03,763 INFO
org.apache.hadoop.hbase.master.ServerManager: Region server
10.146.165.80:60020: MSG_REPORT_EXITING -- lease cancelled


Thanks
-Ankur

-----Original Message-----
From: stack [mailto:stack@duboce.net] 
Sent: Wednesday, March 12, 2008 1:44 AM
To: hbase-user@hadoop.apache.org
Subject: Re: Concurrent Modification Exception

Can we have the stack trace and version number?
Thanks,
St.Ack

Goel, Ankur wrote:
> Hi ,
>      I have a single machine setup running Hadoop and HBase.
> I coupled HBase with Hadoop in order to run the HBase performance 
> evaluation test plan.
>  
> The test plan ran for about 15-20 min and then terminated abruptly 
> taking down the region server along with it.
>  
> Now everytime I try to delete the "TestTable" (created in the failed
> test)
> through the my Java code,  the region server dies and I get a 
> Concurrent
>
> modification exception in the logs. 
>  
> how can I restore the sanity of HBase?
>  
> Thanks
> -Ankur
>
>   


Mime
View raw message