hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Silberstein <silbe...@yahoo-inc.com>
Subject Re: Java client problem w/ zookeeper
Date Wed, 13 Jan 2010 19:43:21 GMT
I have version 0.20.2.  The client requests hang.
And, I looked in the region server logs...here is the likely culprit:

java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for /hbase/root-region-server
        at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKee
perWrapper.java:332)
        at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddress(ZooKeeperWrap
per.java:318)
        at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readRootRegionLocation(Zo
oKeeperWrapper.java:231)
        at 
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:44
2)
        at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for /hbase/root-region-server
        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892)
        at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKee
perWrapper.java:328)
        ... 4 more
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for /hbase/root-region-server
        at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKee
perWrapper.java:332)
        at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddress(ZooKeeperWrap
per.java:318)
        at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readRootRegionLocation(Zo
oKeeperWrapper.java:231)
        at 
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:44
2)
        at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for /hbase/root-region-server
        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892)
        at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKee
perWrapper.java:328)
        ... 4 more

Is there zk state on the region servers?  Maybe when wiping the system, I
should have wiped something on them.

Thanks,
Adam

On 1/13/10 11:33 AM, "Jean-Daniel Cryans" <jdcryans@apache.org> wrote:

> Those node exists are usually harmless, I would like the see what's on
> the HBase region server and master side.
> 
> You say it cannot do any operation but you don't see exceptions
> client-side? It just hangs? Which version is it?
> 
> Thx,
> 
> J-D
> 
> On Wed, Jan 13, 2010 at 11:25 AM, Adam Silberstein
> <silberst@yahoo-inc.com> wrote:
>> Hi,
>> I set up a new Hbase cluster and am seeing an error I haven't seen before.
>> When I start up Hbase I have this NodeExists error in the zookeeper log:
>> 
>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
>> NodeExists
>>        at
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcess
>> or.java:245)
>>        at
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.ja
>> va:114)
>> 2010-01-13 11:10:47,503 WARN
>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>> processing sessionid:0x1262918396b0002 type:create cxid:0xc
>> zxid:0xfffffffffffffffe txntype:unknown n/a
>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
>> NodeExists
>>        at
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcess
>> or.java:245)
>>        at
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.ja
>> va:114)
>> 
>> I've read that may be harmless.  But then when I run the java client, it
>> cannot do any operations.  And I see a similar NodeExists error in the
>> zookeeper log.
>> 
>> 
>> 2010-01-13 11:11:24,661 INFO org.apache.zookeeper.server.NIOServerCnxn:
>> Connected to /203.104.21.210:36928 lastZxid 0
>> 2010-01-13 11:11:24,661 INFO org.apache.zookeeper.server.NIOServerCnxn:
>> Creating new session 0x1262918396b0004
>> 2010-01-13 11:11:24,664 INFO org.apache.zookeeper.server.NIOServerCnxn:
>> Finished init of 0x1262918396b0004 valid:true
>> 2010-01-13 11:11:24,973 WARN
>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>> processing sessionid:0x1262918396b0004 type:create cxid:0x2
>> zxid:0xfffffffffffffffe txntype:unknown n/a
>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
>> NodeExists
>>        at
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcess
>> or.java:245)
>>        at
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.ja
>> va:114)
>> 2010-01-13 11:11:33,088 WARN org.apache.zookeeper.server.NIOServerCnxn:
>> Exception causing close of session 0x1262918396b0004 due to
>> java.io.IOException: Read error
>> 2010-01-13 11:11:33,089 INFO org.apache.zookeeper.server.NIOServerCnxn:
>> closing session:0x1262918396b0004 NIOServerCnxn:
>> java.nio.channels.SocketChannel[connected local=/203.104.21.210:2181
>> remote=/203.104.21.210:36928]
>> 2010-01-13 11:11:42,709 INFO org.apache.zookeeper.server.NIOServerCnxn:
>> Connected to /203.104.21.212:54837 lastZxid 0
>> 2010-01-13 11:11:42,709 INFO org.apache.zookeeper.server.NIOServerCnxn:
>> Creating new session 0x1262918396b0005
>> 2010-01-13 11:11:42,711 INFO org.apache.zookeeper.server.NIOServerCnxn:
>> Finished init of 0x1262918396b0005 valid:true
>> 2010-01-13 11:11:42,711 WARN
>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>> processing sessionid:0x1262918396b0005 type:create cxid:0x1
>> zxid:0xfffffffffffffffe txntype:unknown n/a
>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
>> NodeExists
>>        at
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcess
>> or.java:245)
>>        at
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.ja
>> va:114)
>> 2010-01-13 11:12:06,002 INFO org.apache.zookeeper.server.SessionTrackerImpl:
>> Expiring session 0x1262918396b0004
>> 2010-01-13 11:12:06,002 INFO org.apache.zookeeper.server.ZooKeeperServer:
>> Expiring session 0x1262918396b0004
>> 2010-01-13 11:12:06,002 INFO
>> org.apache.zookeeper.server.PrepRequestProcessor: Processed session
>> termination request for id: 0x1262918396b0004
>> 
>> I've tried wiping the whole setup, including the zookeeper files that Hbase
>> puts in /tmp and formatting HDFS.  But I get the same problem again.  I'm
>> not seeing any exceptions on the client-side.
>> 
>> Thanks,
>> Adam
>> 
>> 


Mime
View raw message