hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oussama Jilal <jilal.ouss...@gmail.com>
Subject HBase on EMR disk space issue
Date Mon, 22 Jul 2013 14:44:31 GMT
Hello, I need some help with my issue,

We have been running HBase on Amazon EMR (v 0.92) for quite some months, 
today we had a serious problem, Hbase was throwing some "connection 
refused" errors in out client applications, after some investigation, we 
found out that one of the two EC2 instances running hbase (the 
slave/regionserver one) was disk full (800+ G), we can connect to both 
of the instances, but when we try to run a command from the HBase Shell 
(list, scan, disable, truncate ... anything) we only get errors like :

13/07/22 14:06:29 INFO httpclient.HttpMethodDirector: I/O exception 
(java.net.ConnectException) caught when processing request: Connection 
refused
13/07/22 14:06:29 INFO httpclient.HttpMethodDirector: Retrying request

We want to truncate some tables and gz compress some column families but 
we can't do any operation.

Executing the command "jps" on the master returns :

6966 Jps
1968 JobTracker
2177 HQuorumPeer
1814 NameNode
4623 HMaster

but on the region server it only return :

5986 Jps

Rebooting didn't help ...

Any one can help ? We are not very experienced with HBase, especially 
administration.

Thank you.

Mime
View raw message