hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kang Minwoo <minwoo.k...@outlook.com>
Subject Re: Is there any way to check the system stop is requested in performCompaction over time?
Date Tue, 17 Mar 2020 04:15:08 GMT
Additional information)
I am currently using hbase-1, but I am preparing to upgrade to hbase-2.

2020. 3. 17. 11:51, Kang Minwoo <minwoo.kang@outlook.com<mailto:minwoo.kang@outlook.com>>
작성:

Thank you for kindly reply.

I think the solution you gave me is really good.

I didn't know that before, so I took a different approach.
I made a new configuration to check that a system is stopped by time.
And I create a issue[1] and attach a patch[2].

I am not sure this is a proper approach.
I'd appreciate your advice.

[1]: https://issues.apache.org/jira/browse/HBASE-23968
[2]: https://github.com/apache/hbase/pull/1274/files


2020. 3. 17. 01:55, Stack <stack@duboce.net<mailto:stack@duboce.net><mailto:stack@duboce.net>>
작성:

Thanks for the detail below. Helped.

In your custom scanner, can you check RegionCoprocessorEnvironment on each next invocation?
RCE gives you access to the hosting RegionServer via

/** @return reference to the region server services */
RegionServerServices getRegionServerServices();

You can ask RegionServerServices... if it is stopping [2].

(This facility is removed in hbase2 it seems but is available to you in hbase1).

S

2. https://github.com/saintstack/hbase/blob/branch-1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java#L51



On Sun, Mar 1, 2020 at 8:49 PM Kang Minwoo <minwoo.kang@outlook.com<mailto:minwoo.kang@outlook.com><mailto:minwoo.kang@outlook.com>>
wrote:
HBase version is 1.2.9
The region was closed after a few minutes of about 1min.

I use a coprocessor that implements BaseRegionObserver.
The purpose is for deleting data with an expired retention period.

My coprocessor return Custom scanner in BaseRegionObserver.preCompact and when scan type is
COMPACT_DROP_DELETES.
The custom scanner does only check row key (row key contain time) and if row key range is
expired retention period, scanner returns empty List<Cell> result.

While the region does performCompaction[1], the region got a request that is a close region.
writesEnabled is changed false. and then performCompaction is stopped only if writing bytes
is over hbase.hstore.close.check.interval config[2].

In my case, there are too many expired retention period data. therefore the scanner returns
nothing. so writing bytes is zero. It occurs pending close.

The reason why I do not use a TTL. Because the retention period changed dynamically by users.

[1]: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java#L367
[2]: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java#L420-L429

Best regards,
Minwoo Kang

________________________________________
보낸 사람: Stack <stack@duboce.net<mailto:stack@duboce.net><mailto:stack@duboce.net>>
보낸 날짜: 2020년 2월 29일 토요일 02:07
받는 사람: Hbase-User
제목: Re: Is there any way to check the system stop is requested in performCompaction over
time?

On Mon, Feb 24, 2020 at 8:20 PM Kang Minwoo <minwoo.kang@outlook.com<mailto:minwoo.kang@outlook.com><mailto:minwoo.kang@outlook.com>>
wrote:

Hello Users.

Is there any way to check the system stop is requested in
performCompaction over time?

When the region got a close request, the region should wait there is no
compaction and flush.
However, in performCompaction method checked periodically only by write
bytes.
If write bytes is too small or scanner returns empty cells, the region
status persists pending close.

Best regards,
Minwoo Kang



Please provide a bit more context. Link to code. Version. Is Region not
closing?
Thanks,
S

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message