hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Angus He <angu...@gmail.com>
Subject Re: Scan cause too many connection
Date Fri, 30 Jul 2010 03:33:52 GMT
1. try to close the scanner explictly?

2. I do not think HBase will issue a new connection for each StoreFile
for the scan operation.

2010/7/29 baggio liu <baggioss@gmail.com>:
> Hi all,
>    We have 53 machines in our hbase cluster and run 6 clients to scan a
> table. During scanning, we found when a region is scanning , it'll new
> StoreFile object, create a connection to datanode (in fact , create HFile
> Reader), so the number of connection increases by the number of store files.
> We have many store files ,(cross several regions and has not reach minor
> compaction thredshould), too many connections has been created. And after
> scanning, the connection will not closed.
>    As the result of it , a machine which act as region server has too high
> system CPU, and hung for a long time.
>    My question is :
>   1. Why we don't close connection ( in fact, we don't close
> HFile.Reader)after we complete to scan table ? We wanna to reuse connection
> in the next scan ??
>   2. How can we limit the connection number ?
> Thanks & Best regards
> LiuJinglong


View raw message