phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-2478) Rows committed in transaction overlapping index creation are not populated
Date Fri, 18 Dec 2015 17:50:47 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15064314#comment-15064314
] 

James Taylor commented on PHOENIX-2478:
---------------------------------------

Here's a solutions that [~poornachandra] and [~anew] came up with that doesn't require any
changes to Tephra and does the necessary coordination between a CREATE INDEX DDL call and
a DML on the same table to ensure all rows are indexed:

CREATE INDEX
* Start tx1
** Create the index which will add the index row to SYSTEM.CATALOG table at the tx1 write
ptr.
* Commit tx1
* Start tx2
** For all in-flight transactions (which is retrievable from the tx context), add to change
set row keys of the form <txID><table name> and <txID><physical table
name> (if physical table name is different than table name for the case of indexes on views).
* Commit tx2 and retry until successful which means that we were able to commit without conflicts
from an in progress DML (see below for row keys added at DML time).
* Initiate the initial index population for all rows older than the tx2 write pointer

DML
* Start tx (as done today)
** Add to change set a row key of the form <txID><table name> and <txID><physical
table name> (if physical table name is different than table name).
* Commit tx
** On TransactionFailureException, sync our meta data with the server metadata and if our
table has changed (i.e. has a new index), then
*** Abort the transaction
*** Rerun the commit (which will now include index rows for new indexes) which could potentially
cause the same logic to repeat if another index happened to be added.

> Rows committed in transaction overlapping index creation are not populated
> --------------------------------------------------------------------------
>
>                 Key: PHOENIX-2478
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2478
>             Project: Phoenix
>          Issue Type: Sub-task
>            Reporter: James Taylor
>            Assignee: James Taylor
>             Fix For: 4.7.0
>
>
> For a reproducible case, see IndexIT.testCreateIndexAfterUpsertStarted() and the associated
FIXME comments for PHOENIX-2446.
> The case that is failing is when a commit starts before an index exists, but commits
after the index build is completed. For transactional data, this is problematic because the
index gets a timestamp after the commit of the data table mutation and thus these mutations
won't be seen during the commit. Also, when the index is being built, the data hasn't yet
been committed and thus won't be part of the initial index build.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message