phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
Date Mon, 25 Jun 2018 15:01:00 GMT


ASF GitHub Bot commented on PHOENIX-3534:

Github user twdsilva commented on a diff in the pull request:
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/replication/
    @@ -35,20 +35,18 @@
      * during cluster upgrades. However, tenant-owned data such as tenant-owned views need
      * be copied. This WALEntryFilter will only allow tenant-owned rows in SYSTEM.CATALOG
      * be replicated. Data from all other tables is automatically passed. It will also copy
    - * child links in SYSTEM.CATALOG that are globally-owned but point to tenant-owned views.
    + * child links in SYSTEM.CHILD_LINK that are globally-owned but point to tenant-owned
     public class SystemCatalogWALEntryFilter implements WALEntryFilter {
    -  private static byte[] CHILD_TABLE_BYTES =
    -      new byte[]{PTable.LinkType.CHILD_TABLE.getSerializedValue()};
       public WAL.Entry filter(WAL.Entry entry) {
    -    //if the WAL.Entry's table isn't System.Catalog, it auto-passes this filter
    +    //if the WAL.Entry's table isn't System.Catalog or System.Child_Link, it auto-passes
this filter
         //TODO: when Phoenix drops support for pre-1.3 versions of HBase, redo as a WALCellFilter
    -    if (!SchemaUtil.isMetaTable(entry.getKey().getTablename().getName())){
    +    byte[] tableName = entry.getKey().getTablename().getName();
    +	if (!SchemaUtil.isMetaTable(tableName) && !SchemaUtil.isChildLinkTable(tableName)){
    --- End diff --
    SYSTEM.CHILD_LINK contains the parent->child linking rows and cells we use to detect
race conditions (eg a column of conflicting type being added at the same time to a parent
and child). 
    The latter cells are written with a short TTL. 
    I think we can use HBase replication for SYSTEM.CHILD_LINK. All the tenant specific view
metadata rows in SYSTEM.CATALOG start with tenant id. 
    I will modify this filter to how it was before PHOENIX-4229. 
    @gjacoby126  Thanks for the suggestion.

> Support multi region SYSTEM.CATALOG table
> -----------------------------------------
>                 Key: PHOENIX-3534
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: Thomas D'Silva
>            Priority: Major
>         Attachments: PHOENIX-3534-wip.patch
> Currently Phoenix requires that the SYSTEM.CATALOG table is single region based on the
server-side row locks being held for operations that impact a table and all of it's views.
For example, adding/removing a column from a base table pushes this change to all views.
> As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), when a new
table is created we can do a lazy cleanup  of any rows that may be left over from a failed
DDL call (kudos to [~lhofhansl] for coming up with this idea). To implement this efficiently,
we'd need to also do PHOENIX-2051 so that we can efficiently find derived views.
> The implementation would rely on an optimistic concurrency model based on checking our
sequence numbers for each table/view before/after updating. Each table/view row would be individually
locked for their change (metadata for a view or table cannot span regions due to our split
policy), with the sequence number being incremented under lock and then returned to the client.

This message was sent by Atlassian JIRA

View raw message