hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Starting HBase in fully distributed mode...
Date Tue, 08 Dec 2009 20:32:48 GMT
Our scripts do not directly support this at this time but, if the ability to
boot from EBS volumes is implemented (in part) as extended command line
options for ec2-run-instances, then the scripts can be easily modified.


Using EBS to back HDFS is something planned: https://issues.apache.org/jira/browse/HBASE-1980

Currently HDFS is backed with instance storage. So, this assumes that no more than N-1 (where

N is the configured HDFS replication level, default 3) instances go down at any one time.


On the other hand, it is not difficult to shut down HBase, do Hadoop
distcp to copy the HBase filesystem up into S3, and then terminate your cluster. To restore,
once a new cluster is up and running, shut down hbase, delete the rootdir, do the inverse
distcp to restore your data from S3 into a new HBase rootdir, then start HBase. Or, use another
backup and restore method, perhaps an application specific
importer-exporter map reduce job.

   - Andy



________________________________
From: Vaibhav Puranik <vpuranik@gmail.com>
To: hbase-user@hadoop.apache.org
Sent: Tue, December 8, 2009 2:32:29 PM
Subject: Re: Starting HBase in fully distributed mode...

Something,

You can now boot your instance with an EBS  volume. This is different from
attaching an EBS volume after booting up an instance.
This will simplify your setup. Furthermore, this functionality also offers a
new command to stop an instance which is different from 'Terminate'. Thus
you can stop your instance but never loose data.

Regards,
Vaibhav Puranik
Gumgum

On Tue, Dec 8, 2009 at 11:11 AM, Something Something <
mailinglists19@gmail.com> wrote:

> Awesome.  Does it allow me to create HDFS on *my* EBS volumes?  If not, I
> probably can change that later.  Without EBS volumes all my data would be
> lost after I terminate my instances, right?
>
> On Tue, Dec 8, 2009 at 10:00 AM, Andrew Purtell <apurtell@apache.org>
> wrote:
>
> > Use the HBase EC2 scripts to launch and manage a cluster up in EC2. The
> > scripts launch the instances themselves, and take care of details such
> > as setting up security groups, allocating and starting a dedicated ZK
> > quorum ensemble, adding ZK and master private DNS names to the config
> > and distributing it to the slaves, etc.
> >
> > See subtasks on HBASE-1961 for further developments planned.
> >
> >   - Andy
> >
> >
> >
> >
> > ________________________________
> > From: Something Something <mailinglists19@gmail.com>
> > To: hbase-user@hadoop.apache.org
> > Sent: Tue, December 8, 2009 12:40:17 PM
> > Subject: Re: Starting HBase in fully distributed mode...
> >
> > Cool.  Didn't know there were scripts in the trunk for HBase.  Will
> > definitely try those.
> >
> > Should I first run the scripts for Hadoop (from <Hadoop>/src/contrib/ec2)
> > and then run scripts for HBase (from <HBase>/src/contrib/ec2)?  Or would
> > the
> > HBase scripts install/format HDFS automatically for me so that I don't
> need
> > to install Hadoop separately?
> >
> > In other words, what should be my approach:
> >
> > 1)  Launch 3 instances on EC2.  Run HBase scripts against them.
> >
> > OR
> >
> > 2)  Launch 3 instances on EC2. Install Hadoop.  Format HDFS.  Start HDFS.
> > Run HBase scripts against these instances?
> >
> > Please let me know.  Thanks.
> >
> >
> > On Tue, Dec 8, 2009 at 8:50 AM, Andrew Purtell <apurtell@apache.org>
> > wrote:
> >
> > > >> Good advice. This is what our EC2 scripts -- in src/contrib/ec2/ --
> > > >> do, and so I can say it works.
> > > >
> > > > I don't think there are scripts for HBase on EC2, are there?
> > >
> > > We (HBase) have EC2 scripts -- src/contrib/ec2/, in SVN, in trunk and
> on
> > > the 0.20 branch.
> > >
> > >   - Andy
> > >
> > >
> > >
> >
> >
> >
> >
> >
>



      
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message