phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chenglei (Jira)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-5996) IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly delete index row when a delete and put mutation with the same timestamp
Date Thu, 09 Jul 2020 07:39:00 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

chenglei updated PHOENIX-5996:
------------------------------
    Description: 
With PHOENIX-5748, {{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}} is responsible
for generating index table mutations for rebuild.
In the processing of data table mutations list, there can be a delete and put mutation with
the same timestamp.   If so, the delete and put are processed together in one iteration. First,
the delete mutation is applied on the put mutation and current row state, and then the modified
put mutation is processed.
But when the {{modified put mutation}} is empty , even the current row state is not empty
after the delete mutation is applied, the whole index row is  deleted, just as following line
1191 in {{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}}:
{code:java}
1189              } else {
1190                    if (currentDataRowState != null) {
1191                        Mutation del = indexMaintainer.buildRowDeleteMutation(indexRowKeyForCurrentDataRow,
1192                                IndexMaintainer.DeleteType.ALL_VERSIONS, ts);
1193                        indexMutations.add(del);
1194                        // For the next iteration of the for loop
1195                        currentDataRowState = null;
1196                        indexRowKeyForCurrentDataRow = null;
1197                    }
1198              }
{code} 

I think above logical is wrong, when the current row state is not empty after the delete mutation
is applied, we can not  delete the whole index row, but instead we should reuse the logical
of processing a delete mutation.  I wrote a unit test in {{PrepareIndexMutationsForRebuildTest}}
to produce the case:
{code:java}
    @Test
    public void testPutDeleteOnSameTimeStampAndPutNullifiedByDelete() throws Exception {
        SetupInfo info = setup(
                TABLE_NAME,
                INDEX_NAME,
                "ROW_KEY VARCHAR, CF1.C1 VARCHAR, CF2.C2 VARCHAR",
                "CF2.C2",
                "ROW_KEY",
                "");

        Put dataPut = new Put(Bytes.toBytes(ROW_KEY));
        addCellToPutMutation(
                dataPut,
                Bytes.toBytes("CF2"),
                Bytes.toBytes("C2"),
                1,
                Bytes.toBytes("v2"));
        addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 1);
        
        addCellToPutMutation(
                dataPut,
                Bytes.toBytes("CF1"),
                Bytes.toBytes("C1"),
                2,
                Bytes.toBytes("v1"));
        addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 2);

        Delete dataDel = new Delete(Bytes.toBytes(ROW_KEY));
        addCellToDelMutation(
                dataDel,
                Bytes.toBytes("CF1"),
                null,
                2,
                KeyValue.Type.DeleteFamily);

        List<Mutation> actualIndexMutations = IndexRebuildRegionScanner.prepareIndexMutationsForRebuild(
                info.indexMaintainer,
                dataPut,
                dataDel);

        List<Mutation> expectedIndexMutations = new ArrayList<>();
        byte[] idxKeyBytes = generateIndexRowKey("v2");

        
        Put idxPut1 = new Put(idxKeyBytes);
        addEmptyColumnToIndexPutMutation(idxPut1, info.indexMaintainer, 1);
        expectedIndexMutations.add(idxPut1);
        
        Put idxPut2 = new Put(idxKeyBytes);
        addEmptyColumnToIndexPutMutation(idxPut2, info.indexMaintainer, 2);
        expectedIndexMutations.add(idxPut2);

        assertEqualMutationList(expectedIndexMutations, actualIndexMutations);
    }
{code} 


  was:With PHOENIX-5748, IndexRebuildRegionScanner is responsible for generating index table
mutations


> IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly delete index
row when a delete and put mutation with the same timestamp
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-5996
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5996
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 5.1.0, 4.16.0
>            Reporter: chenglei
>            Priority: Major
>
> With PHOENIX-5748, {{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}} is responsible
for generating index table mutations for rebuild.
> In the processing of data table mutations list, there can be a delete and put mutation
with the same timestamp.   If so, the delete and put are processed together in one iteration.
First, the delete mutation is applied on the put mutation and current row state, and then
the modified put mutation is processed.
> But when the {{modified put mutation}} is empty , even the current row state is not empty
after the delete mutation is applied, the whole index row is  deleted, just as following line
1191 in {{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}}:
> {code:java}
> 1189              } else {
> 1190                    if (currentDataRowState != null) {
> 1191                        Mutation del = indexMaintainer.buildRowDeleteMutation(indexRowKeyForCurrentDataRow,
> 1192                                IndexMaintainer.DeleteType.ALL_VERSIONS, ts);
> 1193                        indexMutations.add(del);
> 1194                        // For the next iteration of the for loop
> 1195                        currentDataRowState = null;
> 1196                        indexRowKeyForCurrentDataRow = null;
> 1197                    }
> 1198              }
> {code} 
> I think above logical is wrong, when the current row state is not empty after the delete
mutation is applied, we can not  delete the whole index row, but instead we should reuse the
logical of processing a delete mutation.  I wrote a unit test in {{PrepareIndexMutationsForRebuildTest}}
to produce the case:
> {code:java}
>     @Test
>     public void testPutDeleteOnSameTimeStampAndPutNullifiedByDelete() throws Exception
{
>         SetupInfo info = setup(
>                 TABLE_NAME,
>                 INDEX_NAME,
>                 "ROW_KEY VARCHAR, CF1.C1 VARCHAR, CF2.C2 VARCHAR",
>                 "CF2.C2",
>                 "ROW_KEY",
>                 "");
>         Put dataPut = new Put(Bytes.toBytes(ROW_KEY));
>         addCellToPutMutation(
>                 dataPut,
>                 Bytes.toBytes("CF2"),
>                 Bytes.toBytes("C2"),
>                 1,
>                 Bytes.toBytes("v2"));
>         addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 1);
>         
>         addCellToPutMutation(
>                 dataPut,
>                 Bytes.toBytes("CF1"),
>                 Bytes.toBytes("C1"),
>                 2,
>                 Bytes.toBytes("v1"));
>         addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 2);
>         Delete dataDel = new Delete(Bytes.toBytes(ROW_KEY));
>         addCellToDelMutation(
>                 dataDel,
>                 Bytes.toBytes("CF1"),
>                 null,
>                 2,
>                 KeyValue.Type.DeleteFamily);
>         List<Mutation> actualIndexMutations = IndexRebuildRegionScanner.prepareIndexMutationsForRebuild(
>                 info.indexMaintainer,
>                 dataPut,
>                 dataDel);
>         List<Mutation> expectedIndexMutations = new ArrayList<>();
>         byte[] idxKeyBytes = generateIndexRowKey("v2");
>         
>         Put idxPut1 = new Put(idxKeyBytes);
>         addEmptyColumnToIndexPutMutation(idxPut1, info.indexMaintainer, 1);
>         expectedIndexMutations.add(idxPut1);
>         
>         Put idxPut2 = new Put(idxKeyBytes);
>         addEmptyColumnToIndexPutMutation(idxPut2, info.indexMaintainer, 2);
>         expectedIndexMutations.add(idxPut2);
>         assertEqualMutationList(expectedIndexMutations, actualIndexMutations);
>     }
> {code} 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message