hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From baggio liu <baggi...@gmail.com>
Subject Scan cause too many connection
Date Thu, 29 Jul 2010 02:09:56 GMT
Hi all,
    We have 53 machines in our hbase cluster and run 6 clients to scan a
table. During scanning, we found when a region is scanning , it'll new
StoreFile object, create a connection to datanode (in fact , create HFile
Reader), so the number of connection increases by the number of store files.
We have many store files ,(cross several regions and has not reach minor
compaction thredshould), too many connections has been created. And after
scanning, the connection will not closed.
    As the result of it , a machine which act as region server has too high
system CPU, and hung for a long time.
    My question is :
   1. Why we don't close connection ( in fact, we don't close
HFile.Reader)after we complete to scan table ? We wanna to reuse connection
in the next scan ??
   2. How can we limit the connection number ?

Thanks & Best regards
LiuJinglong

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message