kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ale...@apache.org
Subject [1/2] kudu git commit: Optionally advance safe time with non-write operations
Date Sat, 03 Mar 2018 06:23:11 GMT
Repository: kudu
Updated Branches:
  refs/heads/branch-1.6.x 1042e4164 -> e8c68e6c0


Optionally advance safe time with non-write operations

We currently have a problem with safe time advancement when
there are no write messages in the WAL. Because of KUDU-2233,
NO_OP and CHANGE_CONFIG messages are not guaranteed to have
monotonic timestamps, which forced us to ignore them in the
past. This can cause compactions to execute with a timestamp
that is in the beginning of time, causing corruption or crashes.

This patch enables us to advance safe time for all messages
if a flag a set.

Once KUDU-2233 is solved we can flip the flag or remove it
altogether.

Change-Id: I99de2284ba1626581c6e0770c823fd1ba714c1d7
Reviewed-on: http://gerrit.cloudera.org:8080/9396
Tested-by: David Ribeiro Alves <davidralves@gmail.com>
Reviewed-by: Todd Lipcon <todd@apache.org>
(cherry picked from commit 611627f2c2e1137f50373954090fa9f7a76badef)
Reviewed-on: http://gerrit.cloudera.org:8080/9481
Reviewed-by: David Ribeiro Alves <davidralves@gmail.com>
Tested-by: Alexey Serbin <aserbin@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/b9aa69c0
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/b9aa69c0
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/b9aa69c0

Branch: refs/heads/branch-1.6.x
Commit: b9aa69c0d0f4fc6f4778e71bab5290cfbf34dded
Parents: 1042e41
Author: David Alves <dralves@apache.org>
Authored: Mon Jan 8 12:50:00 2018 -0800
Committer: Alexey Serbin <aserbin@cloudera.com>
Committed: Sat Mar 3 06:20:56 2018 +0000

----------------------------------------------------------------------
 src/kudu/tablet/tablet_bootstrap.cc | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/b9aa69c0/src/kudu/tablet/tablet_bootstrap.cc
----------------------------------------------------------------------
diff --git a/src/kudu/tablet/tablet_bootstrap.cc b/src/kudu/tablet/tablet_bootstrap.cc
index 371a6f0..2d17344 100644
--- a/src/kudu/tablet/tablet_bootstrap.cc
+++ b/src/kudu/tablet/tablet_bootstrap.cc
@@ -88,6 +88,10 @@
 #include "kudu/util/pb_util.h"
 #include "kudu/util/stopwatch.h"
 
+DEFINE_bool(advance_safe_time_with_non_write_ops, false,
+            "Whether to advance safe time with committed non-write ops like NO_OP "
+            "or CONFIG_CHANGE. This is not totally safe until KUDU-2233 is fixed.");
+TAG_FLAG(advance_safe_time_with_non_write_ops, experimental);
 
 DECLARE_int32(group_commit_queue_size_bytes);
 
@@ -1079,11 +1083,11 @@ Status TabletBootstrap::HandleEntryPair(LogEntryPB* replicate_entry,
LogEntryPB*
 
 #undef RETURN_NOT_OK_REPLAY
 
-  // Non-tablet operations should not advance the safe time, because they are
-  // not started serially and so may have timestamps that are out of order.
-  if (op_type == NO_OP || op_type == CHANGE_CONFIG_OP) {
-    return Status::OK();
-  }
+   // Non-tablet operations should not advance the safe time until KUDU-2233 is fixed.
+   if (!FLAGS_advance_safe_time_with_non_write_ops &&
+       (op_type == NO_OP || op_type == CHANGE_CONFIG_OP)) {
+     return Status::OK();
+   }
 
   // Handle safe time advancement:
   //
@@ -1092,8 +1096,10 @@ Status TabletBootstrap::HandleEntryPair(LogEntryPB* replicate_entry,
LogEntryPB*
   // safe timestamp to this operation's timestamp.
   //
   // If the hybrid clock is disabled, all transactions will fall into this category.
+  DCHECK(replicate->has_timestamp());
   Timestamp safe_time;
-  if (replicate->write_request().external_consistency_mode() != COMMIT_WAIT) {
+  if (!replicate->write_request().has_external_consistency_mode() ||
+      replicate->write_request().external_consistency_mode() != COMMIT_WAIT) {
     safe_time = Timestamp(replicate->timestamp());
   // ... else we set the safe timestamp to be the transaction's timestamp minus the maximum
clock
   // error. This opens the door for problems if the flags changed across reboots, but this
is


Mime
View raw message