hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kang Minwoo <minwoo.k...@outlook.com>
Subject Re: Is there any way to check the system stop is requested in performCompaction over time?
Date Tue, 17 Mar 2020 02:51:11 GMT
Thank you for kindly reply.

I think the solution you gave me is really good.

I didn't know that before, so I took a different approach.
I made a new configuration to check that a system is stopped by time.
And I create a issue[1] and attach a patch[2].

I am not sure this is a proper approach.
I'd appreciate your advice.

[1]: https://issues.apache.org/jira/browse/HBASE-23968
[2]: https://github.com/apache/hbase/pull/1274/files


2020. 3. 17. 01:55, Stack <stack@duboce.net<mailto:stack@duboce.net>> 작성:

Thanks for the detail below. Helped.

In your custom scanner, can you check RegionCoprocessorEnvironment on each next invocation?
RCE gives you access to the hosting RegionServer via

/** @return reference to the region server services */
RegionServerServices getRegionServerServices();

You can ask RegionServerServices... if it is stopping [2].

(This facility is removed in hbase2 it seems but is available to you in hbase1).

S

2. https://github.com/saintstack/hbase/blob/branch-1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java#L51



On Sun, Mar 1, 2020 at 8:49 PM Kang Minwoo <minwoo.kang@outlook.com<mailto:minwoo.kang@outlook.com>>
wrote:
HBase version is 1.2.9
The region was closed after a few minutes of about 1min.

I use a coprocessor that implements BaseRegionObserver.
The purpose is for deleting data with an expired retention period.

My coprocessor return Custom scanner in BaseRegionObserver.preCompact and when scan type is
COMPACT_DROP_DELETES.
The custom scanner does only check row key (row key contain time) and if row key range is
expired retention period, scanner returns empty List<Cell> result.

While the region does performCompaction[1], the region got a request that is a close region.
writesEnabled is changed false. and then performCompaction is stopped only if writing bytes
is over hbase.hstore.close.check.interval config[2].

In my case, there are too many expired retention period data. therefore the scanner returns
nothing. so writing bytes is zero. It occurs pending close.

The reason why I do not use a TTL. Because the retention period changed dynamically by users.

[1]: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java#L367
[2]: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java#L420-L429

Best regards,
Minwoo Kang

________________________________________
보낸 사람: Stack <stack@duboce.net<mailto:stack@duboce.net>>
보낸 날짜: 2020년 2월 29일 토요일 02:07
받는 사람: Hbase-User
제목: Re: Is there any way to check the system stop is requested in performCompaction over
time?

On Mon, Feb 24, 2020 at 8:20 PM Kang Minwoo <minwoo.kang@outlook.com<mailto:minwoo.kang@outlook.com>>
wrote:

> Hello Users.
>
> Is there any way to check the system stop is requested in
> performCompaction over time?
>
> When the region got a close request, the region should wait there is no
> compaction and flush.
> However, in performCompaction method checked periodically only by write
> bytes.
> If write bytes is too small or scanner returns empty cells, the region
> status persists pending close.
>
> Best regards,
> Minwoo Kang
>


Please provide a bit more context. Link to code. Version. Is Region not
closing?
Thanks,
S

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message