hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alok Kumar <alok...@gmail.com>
Subject Re: Hbase bkup options
Date Mon, 23 Jul 2012 16:24:49 GMT
Hello everyone,

I too have similar use-case, where I've setup a separate HBase Replica
and enabled 'Replciation_Scope' for tables.

Q. Do I need to create 'table + ColFamily' in backup cluster everytime a
new *table* gets created in 'production' cluster?
or Is there a way where table schema too get replicated across cluster(
like put+delete get replicated) ?

Your help is highly appreciated

(I tried sending separate email to group, but it get returned as spam :(

On Mon, Jul 23, 2012 at 9:03 PM, Amlan Roy <amlan.roy@cleartrip.com> wrote:

> Hi Michael,
> Thanks a lot for the reply. What I want to achieve is, if my cluster goes
> down for some reason, I should be able to create a new cluster and should
> be
> able to import all the backed up data. As I want to store all the tables, I
> expect the data size to be huge (in order of Tera Bytes) and it will keep
> growing.
> If I have understood correctly, you have suggested to run "export" to get
> the data into hdfs and then run "hadoop fs -copyToLocal" to get it into
> local file. If I take a back up of the files, is it possible to import that
> data to a new Hbase cluster?
> Thanks and regards,
> Amlan
> -----Original Message-----
> From: Michael Segel [mailto:michael_segel@hotmail.com]
> Sent: Monday, July 23, 2012 8:19 PM
> To: user@hbase.apache.org
> Subject: Re: Hbase bkup options
> Amian,
> Like always the answer to your question is... it depends.
> First, how much data are we talking about?
> What's the value of the underlying data?
> One possible scenario...
> You run a M/R job to copy data from the table to an HDFS file, that is then
> copied to attached storage on an edge node and then to tape.
> Depending on how much data, how much disk is in the attached storage you
> may
> want to keep a warm copy there, a 'warmer/hot' copy on HDFS and then a cold
> copy on tape off to some offsite storage facility.
> There are other options, but it all depends on what you want to achieve.
> With respect to the other tools...
> You can export  (which is a m/r job) to a local directory, then use distcp
> to a different cluster.  hadoop fs -copyToLocal will let you copy off the
> cluster.
> You could write your own code, but you don't get much gain over existing
> UNIX/Linux tools.
> On Jul 23, 2012, at 7:52 AM, Amlan Roy wrote:
> > Hi,
> >
> >
> >
> > Is it feasible to do disk or tape backup for Hbase tables?
> >
> >
> >
> > I have read about the tools like Export, CopyTable, Distcp. It seems like
> > they will require a separate HDFS cluster to do that.
> >
> >
> >
> > Regards,
> >
> > Amlan
> >
Alok Kumar

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message