hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <els...@apache.org>
Subject Re: HBase table snapshots compatibility between 1.0 and 2.1
Date Tue, 12 Nov 2019 16:56:38 GMT
Hey Shuai,

You're likely to get some more traction with this question via 
contacting Cloudera's customer support channels. We try to keep this 
forum focused on Apache HBase versions.

If you are not seeing records after restoring, it sounds like there is 
some (missing?) metadata in the old version which is not handled in the 
newer versions.

As far as your procedurces, you could combine your 
create/disable/restore_snapshot to just use clone_snapshot instead. 
However, if there is some incompatibility, this is of little consequence.

You could try to use CopyTable instead of snapshots.

On 11/11/19 1:47 AM, Shuai Lin wrote:
> Hi all,
> TL;DR Could table snapshots taken in hbase 1.0 be used in hbase 2.1?
> We have an existing production hbase 1.0 cluster (CDH 5.4) , and we're
> setting up a new cluster with hbase 2.1 (CDH 6.3). Let's call the old
> cluster C1 and new one C2.
> To migrate the existing data from C1 to C2, we plan to use the "snapshot +
> replication" approach (snapshot would capture the existing part, and
> replication would do the incremental part) . However when I was testing the
> feasibility of this approach locally, I found that the snapshot could be
> successfully export to c2, but but the restored table on C2 has no data.
> Here is a minimal reproducible example:
> 1. on C1: take the snapshot and export it to C2
> hbase shell:
>      create "t1", {"NAME"=>"f1", "REPLICATION_SCOPE" => 1}
>      put "t1", "r1", "f1:c1", "value"
>      snapshot 't1', 't1s1'
> sudo -u hdfs hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -snapshot t1s1 \
>              -copy-to hdfs://c2:8020/hbase -mappers 1
> 2. Then on C2 restore the table
> hbase shell:
>      create "t1", {"NAME"=>"f1", "REPLICATION_SCOPE" => 1}
>      disable "t1"
>      restore_snapshot "t1s1"
>      enable "t1"
>      scan "t1"
> All these steps succeeds, except that the final "scan" command shows no
> data at all. Also worth noting that on the master web ui on C2 it shows the
> table t1 has two regions and one is not assigned - It shall have only one
> region obviously.
> So my question is:  Could table snapshots taken in hbase 1.0 be used in
> hbase 2.1?
> - If yes, anything I'm doing wrong here?
> - If no, is there any workaround? (e.g. performing some preprocessing on
> the snapshot data on hbase 2.1 side before restoring it?)
> If this can't work, the only alternative way to migrate the data is too
> install hbase 1.0 on C2 (so it could use the snapshot from C1), and upgrade
> it to hbase 2.1 after restoring the snapshot. I I'd like to avoid going
> this way as much as possible because it would be too cumbersome.
> Any information would be much appreciated, thx!

View raw message