phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3955) Ensure KEEP_DELETED_CELLS, REPLICATION_SCOPE, and TTL properties stay in sync between the physical data table and index tables
Date Mon, 11 Jun 2018 21:06:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508748#comment-16508748
] 

ASF GitHub Bot commented on PHOENIX-3955:
-----------------------------------------

Github user twdsilva commented on a diff in the pull request:

    https://github.com/apache/phoenix/pull/304#discussion_r194545366
  
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
---
    @@ -767,50 +748,117 @@ private void modifyColumnFamilyDescriptor(HColumnDescriptor hcd,
Map<String,Obje
             }
         }
     
    +    /**
    +     * Check that the specific set of column family properties mentioned in {@link MetaDataUtil#PROPERTIES_TO_KEEP_IN_SYNC_AMONG_COL_FAMS_AND_INDEXES}
    +     * are in sync for the given table (this includes the local index column family).
    +     * @param tableName
    +     * @param defaultFamilyBytes
    +     * @return default column descriptor used for checking if properties are synced or
not, and the synced table descriptor
    +     * @throws SQLException
    +     */
    +    private Pair<HColumnDescriptor, HTableDescriptor> ensureTableColumnFamilyPropsInSync(String
tableName, byte[] defaultFamilyBytes) throws SQLException {
    +        HTableDescriptor tableDescriptor = getTableDescriptor(Bytes.toBytes(tableName));
    +        HColumnDescriptor[] colFamilies = tableDescriptor.getColumnFamilies();
    +        HColumnDescriptor  defaultColDescriptor = tableDescriptor.getFamily(defaultFamilyBytes);
    +        // It's possible that the table has specific column families and none of them
are declared to be the DEFAULT_COLUMN_FAMILY
    +        defaultColDescriptor = defaultColDescriptor != null ? defaultColDescriptor :
colFamilies[0];
    +
    +        for (String propName: MetaDataUtil.PROPERTIES_TO_KEEP_IN_SYNC_AMONG_COL_FAMS_AND_INDEXES)
{
    +            if (defaultColDescriptor.getValue(propName) == null) {
    +                if (!isUpgradeRequired()) {
    +                    // We cannot have a null value for any of the properties that need
to be kept in sync amongst all column families
    +                    logger.error("Cannot have null value for column family property:
" + propName);
    +                    setUpgradeRequired();
    +                    throw new UpgradeRequiredException();
    +                } else {
    +                    defaultColDescriptor.setValue(propName, HColumnDescriptor.getDefaultValues().get(propName));
    +                }
    +            }
    +        }
    +        // Used in the upgrade path to actually fix the table descriptor by syncing properties
    +        HTableDescriptor syncedTableDescriptor = new HTableDescriptor(tableDescriptor);
    +        // Check that these properties are in sync amongst all column families of the
table
    +        for (HColumnDescriptor family: colFamilies) {
    +            if (isUpgradeRequired()) {
    +                family = syncedTableDescriptor.getFamily(family.getName());
    +            }
    +            for (String propName: MetaDataUtil.PROPERTIES_TO_KEEP_IN_SYNC_AMONG_COL_FAMS_AND_INDEXES)
{
    +                if (!family.getValue(propName).toUpperCase(Locale.ROOT).equals(defaultColDescriptor.getValue(propName).toUpperCase(Locale.ROOT)))
{
    --- End diff --
    
    Do you need the toUpperCase(Locale.ROOT) ?


> Ensure KEEP_DELETED_CELLS, REPLICATION_SCOPE, and TTL properties stay in sync between
the physical data table and index tables
> ------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3955
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3955
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Samarth Jain
>            Assignee: Chinmay Kulkarni
>            Priority: Major
>
> We need to make sure that indexes inherit the REPLICATION_SCOPE, KEEP_DELETED_CELLS and
TTL properties from the base table. Otherwise we can run into situations where the data was
removed (or not removed) from the data table but was removed (or not removed) from the index.
Or vice-versa. We also need to make sure that any ALTER TABLE SET TTL or ALTER TABLE SET KEEP_DELETED_CELLS
statements propagate the properties to the indexes too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message