hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lex Toumbourou <...@scrunch.com>
Subject Deleting and cleaning old snapshots exported to S3
Date Tue, 21 Nov 2017 03:45:28 GMT
Hi all,

Wondering if I could get some help figuring out how to clean out old
snapshots that have been exported to S3?

We've been exporting snapshots to S3 using the export snapshot command:

bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
some-snapshot -copy-to s3a://some-bucket/hbase

Now the size of the S3 bucket is getting a little out of control and I'd
like to remove the old snapshots and let HBase garbage collect blocks no
longer referenced.

One idea I had was to spin up an entirely new cluster that uses the S3
bucket as the hbase.rootdir then just delete the snapshots as normal and
maybe use cleaner_run to clean up the old files but it feels like overkill
having to spin up an entire cluster.

So my question is: what's the best approach for deleting snapshots exported
to an s3 bucket and cleaning old store files no longer referenced? We are
using HBase 1.3.1 on EMR.


Lex ToumbourouCTO at scrunch.com <http://scrunch.com/>

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message