phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "daniel meng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-1107) Support mutable indexes over replication
Date Tue, 30 Sep 2014 08:33:33 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152952#comment-14152952
] 

daniel meng commented on PHOENIX-1107:
--------------------------------------

[~jesse_yates] i make a patch v1 for 3.0 branch based on your work. the idea is as below:

first ,  we give IndexedKeyValue a columnFamily to let Replication.visitLogEntryBeforeWrite
pass without NPE.  Both DataTable and IndexTable use same DEFAULT_COLUMN_FAMILY, so we choose
it .

second, replace IndexedKeyValue's columnFamily by "_0-INDEX_NO_REPL" just before ReplicationSource
get the WALEdit. done this through a WALObserver.  

as a result, ReplicationSource.removeNonRelicableEdits will remove IndexedKeyValue ones because
edit.hasKeyInScope("_0-INDEX_NO_REPL") return false 

{code}
  protected void removeNonReplicableEdits(WALEdit edit,byte[] table) {
    List<KeyValue> kvs = edit.getKeyValues();
    for (int i = edit.size()-1; i >= 0; i--) {
      KeyValue kv = kvs.get(i);
      // The scope will be null or empty if
      // there's nothing to replicate in that WALEdit
      if (!edit.hasKeyInScope(kv.getFamily())) {
        kvs.remove(i);
        continue;
      }
      if(this.conf.getBoolean("hbase.replication.enable.filter", false)){
        if(!this.manager.checkIfQualifyIsReplicationable(table, kv.getFamily(), kv.getQualifier())){
          kvs.remove(i);
        }  
      }      
    }
  }
{code}

i also changed the IT, pls have a review, i'm look forward for your advice.

> Support mutable indexes over replication
> ----------------------------------------
>
>                 Key: PHOENIX-1107
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1107
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 5.0.0, 3.1, 4.1
>            Reporter: Jesse Yates
>            Assignee: Jesse Yates
>         Attachments: phoenix-1107-3.0.v0, phoenix-1107-master-passes.patch
>
>
> Mutable indexes don't support usage with replication. For starters, the replication WAL
Listener checks the family of the edits, which can throw a NPE for the IndexedKeyValue 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message