phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "daniel meng (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-1107) Support mutable indexes over replication
Date Tue, 30 Sep 2014 08:33:33 GMT


daniel meng commented on PHOENIX-1107:

[~jesse_yates] i make a patch v1 for 3.0 branch based on your work. the idea is as below:

first ,  we give IndexedKeyValue a columnFamily to let Replication.visitLogEntryBeforeWrite
pass without NPE.  Both DataTable and IndexTable use same DEFAULT_COLUMN_FAMILY, so we choose
it .

second, replace IndexedKeyValue's columnFamily by "_0-INDEX_NO_REPL" just before ReplicationSource
get the WALEdit. done this through a WALObserver.  

as a result, ReplicationSource.removeNonRelicableEdits will remove IndexedKeyValue ones because
edit.hasKeyInScope("_0-INDEX_NO_REPL") return false 

  protected void removeNonReplicableEdits(WALEdit edit,byte[] table) {
    List<KeyValue> kvs = edit.getKeyValues();
    for (int i = edit.size()-1; i >= 0; i--) {
      KeyValue kv = kvs.get(i);
      // The scope will be null or empty if
      // there's nothing to replicate in that WALEdit
      if (!edit.hasKeyInScope(kv.getFamily())) {
      if(this.conf.getBoolean("hbase.replication.enable.filter", false)){
        if(!this.manager.checkIfQualifyIsReplicationable(table, kv.getFamily(), kv.getQualifier())){

i also changed the IT, pls have a review, i'm look forward for your advice.

> Support mutable indexes over replication
> ----------------------------------------
>                 Key: PHOENIX-1107
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 5.0.0, 3.1, 4.1
>            Reporter: Jesse Yates
>            Assignee: Jesse Yates
>         Attachments: phoenix-1107-3.0.v0, phoenix-1107-master-passes.patch
> Mutable indexes don't support usage with replication. For starters, the replication WAL
Listener checks the family of the edits, which can throw a NPE for the IndexedKeyValue 

This message was sent by Atlassian JIRA

View raw message