hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Whiting <je...@qualtrics.com>
Subject Re: HBase & MapReduce & Zookeeper
Date Thu, 28 Jul 2011 16:10:16 GMT
10 connection maximum is too low.  It has been recommended to go up to as many as 2000 connections

in the list.  This doesn't fix your problem but is something you should probably have in your

configuration.

~Jeff

On 7/28/2011 10:00 AM, Stack wrote:
> Try getting the ZooKeeperWatcher from the connection on your way out
> and explicitly shutdown the zk connection (see TestZooKeeper unit test
> for example).
> St.Ack
>
> On Thu, Jul 28, 2011 at 6:01 AM, Andre Reiter<a.reiter@web.de>  wrote:
>> this issue is still not resolved...
>>
>> unfortunatelly calling HConnectionManager.deleteConnection(conf, true);
>> after the MR job is finished, does not close the connection to the zookeeper
>> we have 3 zookeeper nodes
>> by default there is a limit of 10 connections allowed from a single client
>>
>> so after running 30 MR jobs scheduled by our application, we have 30
>> unclosed connections, trying to start a new MR job results in a failure, the
>> connection to the zookeeper ensamble is droped...
>>
>> the work around to restart the whole application after 30 MR jobss is not
>> very elegant... :-(
>>
>>
>>

-- 
Jeff Whiting
Qualtrics Senior Software Engineer
jeffw@qualtrics.com


Mime
View raw message