phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gokcen Iskender (Jira)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-5841) When data columns get TTLed, we need inline index validation to publish a metric for this
Date Tue, 14 Apr 2020 23:01:00 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-5841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gokcen Iskender updated PHOENIX-5841:
-------------------------------------
    Description: 
We do index writes as full row writes. This means all columns keep get re-written to index
and they have a current timestamp.

However, if there is a column that did not get updated for a long time (like Created_By type
of columns that don't change) index inline validation marks these as "Index has extra columns".
We need to publish an extra metric to distinguish these cases since they are expected to be
not matching rows.

  was:
We do index writes as full row writes. This means all columns keep get re-written to index
and they have a current timestamp.

However, if there is a column that did not get updated for a long time (like Created_By type
of columns that don't change) the regular scan that IndexScrutiny uses doesn't return this
column even though we might see it in the raw scan.

IndexScrutiny mark these rows as invalid because data table had TTL columns and index table
have it.

Index inline validation also marks these as "Index has extra columns" if the column got TTLed.


> When data columns get TTLed, we need inline index validation to publish a metric for
this
> -----------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-5841
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5841
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Gokcen Iskender
>            Priority: Major
>
> We do index writes as full row writes. This means all columns keep get re-written to
index and they have a current timestamp.
> However, if there is a column that did not get updated for a long time (like Created_By
type of columns that don't change) index inline validation marks these as "Index has extra
columns". We need to publish an extra metric to distinguish these cases since they are expected
to be not matching rows.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message