kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From a...@apache.org
Subject [kudu] branch master updated: external_mini_cluster: raise start process timeout to 60s
Date Fri, 17 May 2019 22:09:45 GMT
This is an automated email from the ASF dual-hosted git repository.

adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git


The following commit(s) were added to refs/heads/master by this push:
     new 08b916d  external_mini_cluster: raise start process timeout to 60s
08b916d is described below

commit 08b916d4b168d8843d663d8b2805910131c2e3cc
Author: Adar Dembo <adar@cloudera.com>
AuthorDate: Wed May 15 15:46:19 2019 -0700

    external_mini_cluster: raise start process timeout to 60s
    
    After commit 7a6df6aa1 changed SystemNtp::Init to wait using ntp_adjtime(),
    a few test failures showed up with the following:
    
      14:01:46.947 [DEBUG - main] (MiniKuduCluster.java:165) Response: error {
        code: TIMED_OUT
        message: "failed to start masters: Unable to start Master at index 0: Timed out after
30.000ss waiting for process (/tmp/dist-test-taskOjnAIG/build/tsan/bin/kudu-master) to write
info file (/tmp/dist-test-taskOjnAIG/test-tmp/mini-kudu-cluster9136683209203726578/master-0/data/info.pb)"
      }
    
    This was because the kernel's clock lost synchronization for more than 30s,
    at which point the minicluster gave up on the process even though
    SystemNtp::Init was prepared to keep waiting.
    
    Here's a bandaid: extend the minicluster's process starting timeout to 60s,
    so that it matches the default value of --ntp_initial_sync_wait_secs. We
    could tightly couple the two values, but I'm not sure that's worth doing.
    
    Change-Id: Ib4172e5b45a325eab9c8387bcf25c106d31b3328
    Reviewed-on: http://gerrit.cloudera.org:8080/13348
    Tested-by: Adar Dembo <adar@cloudera.com>
    Reviewed-by: Will Berkeley <wdberkeley@gmail.com>
---
 src/kudu/mini-cluster/external_mini_cluster.cc | 4 ++--
 src/kudu/mini-cluster/external_mini_cluster.h  | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/src/kudu/mini-cluster/external_mini_cluster.cc b/src/kudu/mini-cluster/external_mini_cluster.cc
index 0592272..73145ba 100644
--- a/src/kudu/mini-cluster/external_mini_cluster.cc
+++ b/src/kudu/mini-cluster/external_mini_cluster.cc
@@ -111,7 +111,7 @@ ExternalMiniClusterOptions::ExternalMiniClusterOptions()
       hms_mode(HmsMode::NONE),
       enable_sentry(false),
       logtostderr(true),
-      start_process_timeout(MonoDelta::FromSeconds(30)),
+      start_process_timeout(MonoDelta::FromSeconds(60)),
       rpc_negotiation_timeout(MonoDelta::FromSeconds(3)) {
 }
 
@@ -985,7 +985,7 @@ Status ExternalDaemon::StartProcess(const vector<string>& user_flags)
{
   if (!success) {
     ignore_result(p->Kill(SIGKILL));
     return Status::TimedOut(
-        Substitute("Timed out after $0s waiting for process ($1) to write info file ($2)",
+        Substitute("Timed out after $0 waiting for process ($1) to write info file ($2)",
                    opts_.start_process_timeout.ToString(), opts_.exe, info_path));
   }
 
diff --git a/src/kudu/mini-cluster/external_mini_cluster.h b/src/kudu/mini-cluster/external_mini_cluster.h
index d9ed431..e038ec7 100644
--- a/src/kudu/mini-cluster/external_mini_cluster.h
+++ b/src/kudu/mini-cluster/external_mini_cluster.h
@@ -29,7 +29,6 @@
 
 #include <boost/optional/optional.hpp>
 #include <glog/logging.h>
-#include <gtest/gtest_prod.h>
 
 #include "kudu/client/shared_ptr.h"
 #include "kudu/common/common.pb.h"
@@ -168,7 +167,7 @@ struct ExternalMiniClusterOptions {
   // Amount of time that may elapse between the creation of a daemon process
   // and the process writing out its info file.
   //
-  // Default: 30s.
+  // Default: 60s.
   MonoDelta start_process_timeout;
 
   // Parameter for the cluster's RPC messenger: timeout interval after which


Mime
View raw message