hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Angus He <angu...@gmail.com>
Subject Re: HBase 0.20.0rc not close the connection to zookeeper explicitly when closing HTable (and HBaseAdmin)
Date Tue, 04 Aug 2009 07:42:03 GMT
Thanks for the brilliant comments, Ryan.

For each of this not so graceful close, zookeeper will populate its
log file with a WARN record that just likes
2009-08-04 15:15:35,831 WARN
org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of
session 0x122e34dd69b00ad due to java.io.IOException: Read error.

It might be confuse some users of HBase, probably we can put some
information about this in the documentation.


On Tue, Aug 4, 2009 at 2:41 PM, Ryan Rawson<ryanobjc@gmail.com> wrote:
> We should move the clients to a non-active server API, possibly the
> REST one, and avoid using active sessions just for clients.  Something
> to address in 0.21 I think.
>
> As for #2, it is recommended now to run a quorum of zookeeper instead
> of a single one.  This reduces the risk of running out of connections.
>
> Also the code snippet you listed is a little degenerate, we can never
> fully protect ourselves from fork-bomb like code.  Your code snippet
> suggests that:
> - you are creating/closing HTable a lot.  Maybe you shouldn't do that?
>  HTablePool?
> - you have 1024+ tables, and need to access them in one client at one time.
>
> In the mean time, highly consider upgrading to a cluster of 5-7 ZK
> hosts.  For production, you should consider NOT running them on your
> HBase/HDFS/map-reduce nodes.
>
> Good luck!
> -ryan
>
> On Mon, Aug 3, 2009 at 11:00 PM, Angus He<angushe@gmail.com> wrote:
>> Hi All,
>>
>> In HBase 0.20rc, HTable does not explicitly close the connection to
>> zookeeper in HTable::close.
>> It probably could be better.  And in my opinion, it should be for:
>>
>> 1. It is not well-behaved, although zookeeper is able to detect the
>> lost connection after issuing networking I/O operation, .
>> 2. It is easy to get zookeeper server stuck with exceptions like "Too
>> many connections from /0:0:0:0:0     :0:0:1 - max is 30", when user
>> write codes like:
>>                        for (int i = 0; i < 1024; ++i) {
>>                                HTable table = new HTable("foobar");
>>                                table.close();
>>                        }
>>
>> In the current implementation, different HTable instances share the
>> same connection to zookeeper if they have same HBaseConfiguration
>> instance. For this, we cannot close the connection directly in HTable,
>> but probably we could implement HConnection class with
>> reference-counting ability.
>>
>> Any comments?
>>
>> --
>> Regards
>> Angus
>>
>



-- 
Regards
Angus

Mime
View raw message