phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ankit Singhal (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-2249) SYSTEM.STATS not update after region merge occurs.
Date Sun, 06 Dec 2015 11:23:11 GMT


Ankit Singhal commented on PHOENIX-2249:

thanks [~jamestaylor] for the review,

{quote}Looking at this closer, I'm confused because we already delete the stats for the regions
being compacted when the InternalScanner returned from createCompactionScanner is closed (see
calls to StatisticsWriter.deleteStats()).{quote}

This only update the parent region. Daughter regions were never updated.

To be honest, I'm not sure what the HBase life cycle is when a merge occurs (i.e. when admin.mergeRegions()
is called) - does it come through this same preCompact coprocessor hook?.{quote}
Yes, during postMerge (at RegionServerObserver level), we can combine them but we need to
ask the user to add this Observer in hbase configuration and also it will not be necessary
as these regions become stale only after compaction is completed.

Is the merge operation distinct/orthogonal to a compaction? I expected there to be a merge
coprocessor hook in which we'd just concatenate the stats information together (as two regions
are now becoming one, but the content hasn't changed).{quote}
Merge is distinct operation than compaction.CompactSplitThread will initiate the compaction
as soon it sees the reference files(daughter region) in the store. Merge coprocessor hook
will also not necessary as compaction will create a new stats for the new region.

{quote}One high level comment - when we update stats, we collect up all the stats information
in our StatisticsWriter and commit them in bulk so that they're updated atomically.{quote}
Yes agreed, we should do this also atomically with update operation. so moved the code where
we are updating stats of parent region.

PFA, updated patch after incorporating your review comments.

> SYSTEM.STATS not update after region merge occurs.
> --------------------------------------------------
>                 Key: PHOENIX-2249
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>         Environment: Ubuntu server 14.04
> Hadoop 2.6.0
> HBase 1.0.0
> Phoenix 4.4.0-HBase-1.0.0
>            Reporter: Kuan-Po Tseng
>            Assignee: Ankit Singhal
>             Fix For: 4.7.0
>         Attachments: PHOENIX-2249.patch
> When region merge occurs, SYSTEM.STATS does not update, it will leave stale information
behind. And then if region splits on the merged region, this may cause 
> "org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108 (XCL08): Cache
of region boundaries are out of date" 
> after creating parallel scans since the stale information in SYSTEM.STATS will let guide
post list not sorted in ascending order and this will cause scans over regions.

This message was sent by Atlassian JIRA

View raw message