hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Is there any way to check the system stop is requested in performCompaction over time?
Date Mon, 16 Mar 2020 16:55:17 GMT
Thanks for the detail below. Helped.

In your custom scanner, can you check RegionCoprocessorEnvironment on each
next invocation? RCE gives you access to the hosting RegionServer via

/** @return reference to the region server services */
RegionServerServices getRegionServerServices();

You can ask RegionServerServices... if it is stopping [2].

(This facility is removed in hbase2 it seems but is available to you in
hbase1).

S

2.
https://github.com/saintstack/hbase/blob/branch-1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java#L51



On Sun, Mar 1, 2020 at 8:49 PM Kang Minwoo <minwoo.kang@outlook.com> wrote:

> HBase version is 1.2.9
> The region was closed after a few minutes of about 1min.
>
> I use a coprocessor that implements BaseRegionObserver.
> The purpose is for deleting data with an expired retention period.
>
> My coprocessor return Custom scanner in BaseRegionObserver.preCompact and
> when scan type is COMPACT_DROP_DELETES.
> The custom scanner does only check row key (row key contain time) and if
> row key range is expired retention period, scanner returns empty List<Cell>
> result.
>
> While the region does performCompaction[1], the region got a request that
> is a close region. writesEnabled is changed false. and then
> performCompaction is stopped only if writing bytes is over
> hbase.hstore.close.check.interval config[2].
>
> In my case, there are too many expired retention period data. therefore
> the scanner returns nothing. so writing bytes is zero. It occurs pending
> close.
>
> The reason why I do not use a TTL. Because the retention period changed
> dynamically by users.
>
> [1]:
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java#L367
> [2]:
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java#L420-L429
>
> Best regards,
> Minwoo Kang
>
> ________________________________________
> 보낸 사람: Stack <stack@duboce.net>
> 보낸 날짜: 2020년 2월 29일 토요일 02:07
> 받는 사람: Hbase-User
> 제목: Re: Is there any way to check the system stop is requested in
> performCompaction over time?
>
> On Mon, Feb 24, 2020 at 8:20 PM Kang Minwoo <minwoo.kang@outlook.com>
> wrote:
>
> > Hello Users.
> >
> > Is there any way to check the system stop is requested in
> > performCompaction over time?
> >
> > When the region got a close request, the region should wait there is no
> > compaction and flush.
> > However, in performCompaction method checked periodically only by write
> > bytes.
> > If write bytes is too small or scanner returns empty cells, the region
> > status persists pending close.
> >
> > Best regards,
> > Minwoo Kang
> >
>
>
> Please provide a bit more context. Link to code. Version. Is Region not
> closing?
> Thanks,
> S
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message