hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Buttler, David" <buttl...@llnl.gov>
Subject RE: how to restart a hbase cluster
Date Tue, 19 Jul 2011 20:48:45 GMT
They are not really worth posting:

${HBASE_HOME}/bin/hbase-daemon.sh start master
ssh node1 ${HBASE_HOME}/bin/hbase-daemon.sh start regionserver

${HBASE_HOME}/bin/hbase-daemon.sh start zookeeper

I need to set up pdsh on all of my clusters so I have a more consistent and usable way to
distribute commands.  Each cluster seems to have its own way of doing things right now.


-----Original Message-----
From: highpointe [mailto:highpointe3i@gmail.com] 
Sent: Tuesday, July 19, 2011 10:09 AM
To: user@hbase.apache.org
Cc: user@hbase.apache.org
Subject: Re: how to restart a hbase cluster


Would you be willing to post your custom scripts?

Your setup sounds useful for what we are doing.


Sent from my iPhone

On Jul 19, 2011, at 10:49 AM, "Buttler, David" <buttler1@llnl.gov> wrote:

> Hi Stack,
> As a further data point, I always use the hbase-daemon.sh scripts to start/stop HBase.
 I modified the start/stop-hbase.sh scripts so that they don't start/stop zookeeper, and I
have a modified version that I call start/stop-zookeeper.sh.  This allows me to use HBase
to manage zookeeper so I can have a more sane configuration system, but not necessarily stop
zookeeper when I stop HBase, since I use zookeeper for some other stuff too.
> Sometimes the region servers don't die when I want them to, so I have another script
that calls the hbase-daemon.sh stop regionserver script in parallel on all of the machines.
 Only rarely do I have to kill -9 one.  But, as far as I can tell, I have never lost data
doing this.
> Dave
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
> Sent: Tuesday, July 19, 2011 12:11 AM
> To: user@hbase.apache.org
> Subject: Re: how to restart a hbase cluster
> On Tue, Jul 19, 2011 at 12:02 AM, Weihua JIANG <weihua.jiang@gmail.com> wrote:
>> It seems stop-hbase.sh only stops master/backup masters and zookeepers.
> Usually it sends a signal to the master that then sets a flag in
> zookeeper.  When regionservers see this flag, they start to close down
> user-space regions.  When all user-space regions have been closed,
> they the server will close catalog regions.  When a regionserver is
> carrying no regions, it shuts itself down.
> The master waits until all regionservers are down.  It then will go down itself.
> If you have set hbase to manage zookeeper, the last thing done on the
> way out is shutdown the zk ensemble.
> This is how it is supposed to work.
>> So, according to my understanding, region servers shall shutdown
>> itself since it can't find either master or zookeeper.
> Hmm  Don't they keep retrying?
>> But, I made a recent experimentation on our hbase cluster. After 2
>> days of mater/zookeeper shutdown, the region servers are still alive.
> That doesn't seem correct.  Did the cluster come up cleanly?  Or did
> the master go down before regionservers came up?
>> I am not sure whether it is the problem in hbase release or our own
>> problem since our version is a heavy patched one.
>> Then, can I perform hbase cluster in following way?
>> 1. stop master
>> 2. stop master backups
>> 3. stop zookeepers
>> 4. stop region servers
>> The only difference is step #4. If I manually stop down RS, will it
>> affect data integrity? If not, then I can safely performed the steps
>> to shutdown the cluster.
> If a regionserver crashes down rather than shutdown cleanly, it will
> leave its wal logs around.  The master will notice them and replay
> them.  So try not to crash out your regionservers. ./bin/stop-hbase.sh
> should put the regionservers all down cleanly.
> If you do ./bin/hbase-daemon.sh stop regionserver, that'll send the
> process a signal.  It'll run its shutdown signal handler.  I think
> this will bring on a clean shutdown.  See the code to be sure.
> if  clean shutdown, data should be preserved.   Even if its not a
> clean shutdown, as long as the log splitting is allowed complete,
> there should be no data loss even if server is crashed down.
> St.Ack

View raw message