hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Silberstein <silbe...@yahoo-inc.com>
Subject Re: Java client problem w/ zookeeper
Date Wed, 13 Jan 2010 21:17:47 GMT
Yes, I do have that set.  In my case, the quorum is just a single machine,
that is also the Hbase master.  And I'm pretty sure that the client is
reaching it, because as soon as I start my client, I see a bunch of the
"NodeExists" exceptions in the zookeeper log.

-Adam


On 1/13/10 1:07 PM, "Jean-Daniel Cryans" <jdcryans@apache.org> wrote:

> Oh, so HBase is working fine but you have a remote client connectivity
> problem. You gave the good hbase.zookeeper.quorum to HTable?
> 
> J-D
> 
> On Wed, Jan 13, 2010 at 1:04 PM, Adam Silberstein
> <silberst@yahoo-inc.com> wrote:
>> No, nothing at the master, other than a bunch of NodeExists exceptions in
>> the zookeeper log.
>> 
>> Something else I've found is that I can add records to the table through the
>> shell, without problems.  It's only from the java client, on a separate box,
>> that I'm having problems.
>> 
>> Thanks,
>> Adam
>> 
>> 
>> On 1/13/10 12:08 PM, "Jean-Daniel Cryans" <jdcryans@apache.org> wrote:
>> 
>>> It means the root region likely wasn't assigned, anything in the
>>> master about it?
>>> 
>>> J-D
>>> 
>>> On Wed, Jan 13, 2010 at 11:43 AM, Adam Silberstein
>>> <silberst@yahoo-inc.com> wrote:
>>>> I have version 0.20.2.  The client requests hang.
>>>> And, I looked in the region server logs...here is the likely culprit:
>>>> 
>>>> java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
>>>> KeeperErrorCode = NoNode for /hbase/root-region-server
>>>>        at
>>>> 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKe>>>>
e
>>>> perWrapper.java:332)
>>>>        at
>>>> 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddress(ZooKeeperWra>>>>
p
>>>> per.java:318)
>>>>        at
>>>> 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readRootRegionLocation(Z>>>>
o
>>>> oKeeperWrapper.java:231)
>>>>        at
>>>> 
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:4>>>>
4
>>>> 2)
>>>>        at java.lang.Thread.run(Thread.java:619)
>>>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
>>>> KeeperErrorCode = NoNode for /hbase/root-region-server
>>>>        at
>>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
>>>>        at
>>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>>>>        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892)
>>>>        at
>>>> 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKe>>>>
e
>>>> perWrapper.java:328)
>>>>        ... 4 more
>>>> java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
>>>> KeeperErrorCode = NoNode for /hbase/root-region-server
>>>>        at
>>>> 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKe>>>>
e
>>>> perWrapper.java:332)
>>>>        at
>>>> 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddress(ZooKeeperWra>>>>
p
>>>> per.java:318)
>>>>        at
>>>> 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readRootRegionLocation(Z>>>>
o
>>>> oKeeperWrapper.java:231)
>>>>        at
>>>> 
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:4>>>>
4
>>>> 2)
>>>>        at java.lang.Thread.run(Thread.java:619)
>>>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
>>>> KeeperErrorCode = NoNode for /hbase/root-region-server
>>>>        at
>>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
>>>>        at
>>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>>>>        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892)
>>>>        at
>>>> 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKe>>>>
e
>>>> perWrapper.java:328)
>>>>        ... 4 more
>>>> 
>>>> Is there zk state on the region servers?  Maybe when wiping the system,
I
>>>> should have wiped something on them.
>>>> 
>>>> Thanks,
>>>> Adam
>>>> 
>>>> On 1/13/10 11:33 AM, "Jean-Daniel Cryans" <jdcryans@apache.org> wrote:
>>>> 
>>>>> Those node exists are usually harmless, I would like the see what's on
>>>>> the HBase region server and master side.
>>>>> 
>>>>> You say it cannot do any operation but you don't see exceptions
>>>>> client-side? It just hangs? Which version is it?
>>>>> 
>>>>> Thx,
>>>>> 
>>>>> J-D
>>>>> 
>>>>> On Wed, Jan 13, 2010 at 11:25 AM, Adam Silberstein
>>>>> <silberst@yahoo-inc.com> wrote:
>>>>>> Hi,
>>>>>> I set up a new Hbase cluster and am seeing an error I haven't seen
>>>>>> before.
>>>>>> When I start up Hbase I have this NodeExists error in the zookeeper
log:
>>>>>> 
>>>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
>>>>>> =
>>>>>> NodeExists
>>>>>>        at
>>>>>> 
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>>
>> >>
>> s
>>>>>> or.java:245)
>>>>>>        at
>>>>>> 
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>>
>> >>
>> a
>>>>>> va:114)
>>>>>> 2010-01-13 11:10:47,503 WARN
>>>>>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>>>>>> processing sessionid:0x1262918396b0002 type:create cxid:0xc
>>>>>> zxid:0xfffffffffffffffe txntype:unknown n/a
>>>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
>>>>>> =
>>>>>> NodeExists
>>>>>>        at
>>>>>> 
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>>
>> >>
>> s
>>>>>> or.java:245)
>>>>>>        at
>>>>>> 
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>>
>> >>
>> a
>>>>>> va:114)
>>>>>> 
>>>>>> I've read that may be harmless.  But then when I run the java client,
it
>>>>>> cannot do any operations.  And I see a similar NodeExists error
in the
>>>>>> zookeeper log.
>>>>>> 
>>>>>> 
>>>>>> 2010-01-13 11:11:24,661 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>>>> Connected to /203.104.21.210:36928 lastZxid 0
>>>>>> 2010-01-13 11:11:24,661 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>>>> Creating new session 0x1262918396b0004
>>>>>> 2010-01-13 11:11:24,664 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>>>> Finished init of 0x1262918396b0004 valid:true
>>>>>> 2010-01-13 11:11:24,973 WARN
>>>>>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>>>>>> processing sessionid:0x1262918396b0004 type:create cxid:0x2
>>>>>> zxid:0xfffffffffffffffe txntype:unknown n/a
>>>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
>>>>>> =
>>>>>> NodeExists
>>>>>>        at
>>>>>> 
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>>
>> >>
>> s
>>>>>> or.java:245)
>>>>>>        at
>>>>>> 
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>>
>> >>
>> a
>>>>>> va:114)
>>>>>> 2010-01-13 11:11:33,088 WARN org.apache.zookeeper.server.NIOServerCnxn:
>>>>>> Exception causing close of session 0x1262918396b0004 due to
>>>>>> java.io.IOException: Read error
>>>>>> 2010-01-13 11:11:33,089 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>>>> closing session:0x1262918396b0004 NIOServerCnxn:
>>>>>> java.nio.channels.SocketChannel[connected local=/203.104.21.210:2181
>>>>>> remote=/203.104.21.210:36928]
>>>>>> 2010-01-13 11:11:42,709 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>>>> Connected to /203.104.21.212:54837 lastZxid 0
>>>>>> 2010-01-13 11:11:42,709 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>>>> Creating new session 0x1262918396b0005
>>>>>> 2010-01-13 11:11:42,711 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>>>> Finished init of 0x1262918396b0005 valid:true
>>>>>> 2010-01-13 11:11:42,711 WARN
>>>>>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>>>>>> processing sessionid:0x1262918396b0005 type:create cxid:0x1
>>>>>> zxid:0xfffffffffffffffe txntype:unknown n/a
>>>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
>>>>>> =
>>>>>> NodeExists
>>>>>>        at
>>>>>> 
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>>
>> >>
>> s
>>>>>> or.java:245)
>>>>>>        at
>>>>>> 
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>>
>> >>
>> a
>>>>>> va:114)
>>>>>> 2010-01-13 11:12:06,002 INFO
>>>>>> org.apache.zookeeper.server.SessionTrackerImpl:
>>>>>> Expiring session 0x1262918396b0004
>>>>>> 2010-01-13 11:12:06,002 INFO org.apache.zookeeper.server.ZooKeeperServer:
>>>>>> Expiring session 0x1262918396b0004
>>>>>> 2010-01-13 11:12:06,002 INFO
>>>>>> org.apache.zookeeper.server.PrepRequestProcessor: Processed session
>>>>>> termination request for id: 0x1262918396b0004
>>>>>> 
>>>>>> I've tried wiping the whole setup, including the zookeeper files
that
>>>>>> Hbase
>>>>>> puts in /tmp and formatting HDFS.  But I get the same problem again.
 I'm
>>>>>> not seeing any exceptions on the client-side.
>>>>>> 
>>>>>> Thanks,
>>>>>> Adam
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>> 


Mime
View raw message