hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Latham <lat...@davelink.net>
Subject Re: How to change Date Tiered Compaction Promotion Time ?
Date Wed, 04 Mar 2020 16:43:27 GMT
I helped with the date tiered compaction work, but it's been awhile.  I'll
chime in in case it's helpful.

If you merely want to shift the storm to a different time of day, you may
be able to accomplish that by changing the CompactionWindowFactory being
used to slide all the windows by X hours.  You should be able to plug in an
adjusted implementation via the configuration
"hbase.hstore.compaction.date.tiered.window.factory.class".

If instead you want to smooth out the storm, you may need to look at how
compactions are being queued and throttled on the compaction threads.

Good luck, and let us know how it goes!

Dave

On Tue, Mar 3, 2020 at 8:09 PM Kien Truong <duckientruong@gmail.com> wrote:

> Hi all,
>
> We are testing Date Tiered Compaction but encounter a problem with the
> promotion time. In the default configuration, at 00:00 UTC every day, there
> is a compaction storm when all the store files of the previous day is
> promoted and compacted, and possibly all the store files for the previous 4
> days as well. We would like to move this promotion time so that it would
> fall into our offpeak hours. How can we achieve this ?
>
> Thanks,
>
> Kien
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message