fluo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] keith-turner opened a new issue #1024: Improve Fluo wait algorithm
Date Fri, 09 Mar 2018 16:39:47 GMT
keith-turner opened a new issue #1024: Improve Fluo wait algorithm
URL: https://github.com/apache/fluo/issues/1024
 
 
   The algorithm used by the wait command could be improved.  Currently it scans the entire
notification locality group and counts the number of notifications.  
   
    * Stop scanning once a single notification is found.  
    * Not always starting scanning at the beginning of the table.  What if the single active
notification is at the end or in the middle of the table.
    * Minimize amount of scanning
   
   The following algorithm is one possible way to accomplish these goals.
   
   ```java
   long gcTimeStamp;
   
   do{
     //get the tables row ranges in random order
     List<RowRange> tablets = getAndShuffleTableRowRanges()
     //get timestamp for oracle before start scanning for notifications, if changes during
scan means
     //transactions could have executed and created new notifications.
     long gcTimeStamp  = oracle.getTimestamp();
   
     for(RowRange tablet : tablets){
       //scan this row range of table with exponential time backoff until no notifications
seen.
       boolean sawNotifications = scanWhileNotEmpty(tablet)
       if(sawNotifications) {
        //TODO should any additional waiting be done here, or was the waiting done by 
        //scanWhileNotEmpty good enough?
         continue outer; //jumps to beginning of do while loop, starting completely over
       }
     }
   while(!timestampChanged(gcTimestamp))
   ```
   
   The algorithm above focuses on a single section of the table and waits for it to have no
notifications.  I think this approach minimizes the total amount of scanning over the current
approach.  Also this approach could be more responsive, if the max wait time for scanWhileNotEmpty
is capped.  It should be ok to cap that wait time since its only scanning a single tablet
and not the entire table.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message