lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 54 - Still unstable
Date Sat, 12 May 2018 05:19:51 GMT
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/54/

7 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestLeaderElectionZkExpiry

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [Overseer] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.Overseer  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.cloud.Overseer.start(Overseer.java:545)  at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:851)  at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)  at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)  at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)  at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)  at org.apache.solr.cloud.ZkController$1.command(ZkController.java:357)  at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:167)  at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)  at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141)  at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)  at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [Overseer]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.Overseer
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.cloud.Overseer.start(Overseer.java:545)
	at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:851)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:357)
	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:167)
	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141)
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)


	at __randomizedtesting.SeedInfo.seed([68590BA2D6E9217E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:303)
	at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestLeaderElectionZkExpiry

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestLeaderElectionZkExpiry:     1) Thread[id=14272, name=OverseerAutoScalingTriggerThread-72610145639333898-dummy.host.com:8984_solr-n_0000000002, state=RUNNABLE, group=Overseer autoscaling triggers]         at java.lang.ClassLoader.findLoadedClass0(Native Method)         at java.lang.ClassLoader.findLoadedClass(ClassLoader.java:1038)         at java.lang.ClassLoader.loadClass(ClassLoader.java:406)         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)         at org.apache.logging.log4j.core.impl.ThrowableProxy.loadClass(ThrowableProxy.java:613)         at org.apache.logging.log4j.core.impl.ThrowableProxy.toExtendedStackTrace(ThrowableProxy.java:738)         at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:164)         at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:141)         at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:122)         at org.apache.logging.log4j.core.impl.Log4jLogEvent.getThrownProxy(Log4jLogEvent.java:605)         at org.apache.logging.log4j.core.pattern.ExtendedThrowablePatternConverter.format(ExtendedThrowablePatternConverter.java:64)         at org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:38)         at org.apache.logging.log4j.core.layout.PatternLayout$PatternSerializer.toSerializable(PatternLayout.java:334)         at org.apache.logging.log4j.core.layout.PatternLayout.toText(PatternLayout.java:233)         at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:218)         at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58)         at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177)         at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170)         at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161)         at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156)         at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129)         at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120)         at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)         at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448)         at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433)         at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)         at org.apache.logging.log4j.core.config.LoggerConfig.logParent(LoggerConfig.java:439)         at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:434)         at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)         at org.apache.logging.log4j.core.config.LoggerConfig.logParent(LoggerConfig.java:439)         at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:434)         at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)         at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403)         at org.apache.logging.log4j.core.config.DefaultReliabilityStrategy.log(DefaultReliabilityStrategy.java:49)         at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)         at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170)         at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125)         at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108)         at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2002)         at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1974)         at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:319)         at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:146)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestLeaderElectionZkExpiry: 
   1) Thread[id=14272, name=OverseerAutoScalingTriggerThread-72610145639333898-dummy.host.com:8984_solr-n_0000000002, state=RUNNABLE, group=Overseer autoscaling triggers]
        at java.lang.ClassLoader.findLoadedClass0(Native Method)
        at java.lang.ClassLoader.findLoadedClass(ClassLoader.java:1038)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:406)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.apache.logging.log4j.core.impl.ThrowableProxy.loadClass(ThrowableProxy.java:613)
        at org.apache.logging.log4j.core.impl.ThrowableProxy.toExtendedStackTrace(ThrowableProxy.java:738)
        at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:164)
        at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:141)
        at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:122)
        at org.apache.logging.log4j.core.impl.Log4jLogEvent.getThrownProxy(Log4jLogEvent.java:605)
        at org.apache.logging.log4j.core.pattern.ExtendedThrowablePatternConverter.format(ExtendedThrowablePatternConverter.java:64)
        at org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:38)
        at org.apache.logging.log4j.core.layout.PatternLayout$PatternSerializer.toSerializable(PatternLayout.java:334)
        at org.apache.logging.log4j.core.layout.PatternLayout.toText(PatternLayout.java:233)
        at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:218)
        at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58)
        at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177)
        at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170)
        at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161)
        at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156)
        at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129)
        at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120)
        at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
        at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448)
        at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433)
        at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
        at org.apache.logging.log4j.core.config.LoggerConfig.logParent(LoggerConfig.java:439)
        at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:434)
        at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
        at org.apache.logging.log4j.core.config.LoggerConfig.logParent(LoggerConfig.java:439)
        at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:434)
        at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
        at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403)
        at org.apache.logging.log4j.core.config.DefaultReliabilityStrategy.log(DefaultReliabilityStrategy.java:49)
        at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)
        at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170)
        at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125)
        at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108)
        at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2002)
        at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1974)
        at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:319)
        at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:146)
        at java.lang.Thread.run(Thread.java:748)
	at __randomizedtesting.SeedInfo.seed([68590BA2D6E9217E]:0)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/34)={   "replicationFactor":"2",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   "autoCreated":"true",   "shards":{     "shard2":{       "replicas":{         "core_node3":{           "core":"testSplitIntegration_collection_shard2_replica_n3",           "leader":"true",           "SEARCHER.searcher.maxDoc":11,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10007_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":11},         "core_node4":{           "core":"testSplitIntegration_collection_shard2_replica_n4",           "SEARCHER.searcher.maxDoc":11,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10008_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":11}},       "range":"0-7fffffff",       "state":"active"},     "shard1":{       "stateTimestamp":"1526285509818106650",       "replicas":{         "core_node1":{           "core":"testSplitIntegration_collection_shard1_replica_n1",           "leader":"true",           "SEARCHER.searcher.maxDoc":14,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10007_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":14},         "core_node2":{           "core":"testSplitIntegration_collection_shard1_replica_n2",           "SEARCHER.searcher.maxDoc":14,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10008_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":14}},       "range":"80000000-ffffffff",       "state":"inactive"},     "shard1_1":{       "parent":"shard1",       "stateTimestamp":"1526285509848791950",       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{         "core_node10":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_1_replica1",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10007_solr",           "base_url":"http://127.0.0.1:10007/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7},         "core_node9":{           "core":"testSplitIntegration_collection_shard1_1_replica0",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10008_solr",           "base_url":"http://127.0.0.1:10008/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7}}},     "shard1_0":{       "parent":"shard1",       "stateTimestamp":"1526285509848530950",       "range":"80000000-bfffffff",       "state":"active",       "replicas":{         "core_node7":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_0_replica0",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10008_solr",           "base_url":"http://127.0.0.1:10008/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7},         "core_node8":{           "core":"testSplitIntegration_collection_shard1_0_replica1",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10007_solr",           "base_url":"http://127.0.0.1:10007/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7}}}}}

Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/34)={
  "replicationFactor":"2",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testSplitIntegration_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":11,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10007_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":11},
        "core_node4":{
          "core":"testSplitIntegration_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":11,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10008_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":11}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1526285509818106650",
      "replicas":{
        "core_node1":{
          "core":"testSplitIntegration_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":14,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10007_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":14},
        "core_node2":{
          "core":"testSplitIntegration_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":14,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10008_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":14}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1526285509848791950",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10007_solr",
          "base_url":"http://127.0.0.1:10007/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7},
        "core_node9":{
          "core":"testSplitIntegration_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10008_solr",
          "base_url":"http://127.0.0.1:10008/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1526285509848530950",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10008_solr",
          "base_url":"http://127.0.0.1:10008/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7},
        "core_node8":{
          "core":"testSplitIntegration_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10007_solr",
          "base_url":"http://127.0.0.1:10007/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7}}}}}
	at __randomizedtesting.SeedInfo.seed([68590BA2D6E9217E:51D7B2E2F916E880]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:299)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
waitFor not elapsed but produced an event

Stack Trace:
java.lang.AssertionError: waitFor not elapsed but produced an event
	at __randomizedtesting.SeedInfo.seed([68590BA2D6E9217E:B923D204F265253]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:180)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueueBlocking

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
	at __randomizedtesting.SeedInfo.seed([68590BA2D6E9217E:2DF379DB92BB9D0A]:0)
	at java.util.concurrent.FutureTask.get(FutureTask.java:205)
	at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.testDistributedQueueBlocking(TestSimDistributedQueue.java:116)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue:     1) Thread[id=11941, name=sdqtest--2789-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194)         at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167)         at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$1(TestSimDistributedQueue.java:107)         at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$799/1372856567.call(Unknown Source)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$10/1192810417.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue: 
   1) Thread[id=11941, name=sdqtest--2789-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194)
        at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167)
        at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$1(TestSimDistributedQueue.java:107)
        at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$799/1372856567.call(Unknown Source)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$10/1192810417.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
	at __randomizedtesting.SeedInfo.seed([68590BA2D6E9217E]:0)


FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
expected:<8> but was:<10>

Stack Trace:
java.lang.AssertionError: expected:<8> but was:<10>
	at __randomizedtesting.SeedInfo.seed([68590BA2D6E9217E:5A5AF5F6CA1DE79]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:354)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13207 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
   [junit4]   2> 2541111 INFO  (SUITE-TestGenericDistributedQueue-seed#[68590BA2D6E9217E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestGenericDistributedQueue_68590BA2D6E9217E-001/init-core-data-001
   [junit4]   2> 2541111 WARN  (SUITE-TestGenericDistributedQueue-seed#[68590BA2D6E9217E]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
   [junit4]   2> 2541111 INFO  (SUITE-TestGenericDistributedQueue-seed#[68590BA2D6E9217E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 2541113 INFO  (SUITE-TestGenericDistributedQueue-seed#[68590BA2D6E9217E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2541128 INFO  (TEST-TestGenericDistributedQueue.testLocallyOffer-seed#[68590BA2D6E9217E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLocallyOffer
   [junit4]   2> 2541278 INFO  (TEST-TestGenericDistributedQueue.testLocallyOffer-seed#[68590BA2D6E9217E]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLocallyOffer
   [junit4]   2> 2541305 INFO  (TEST-TestGenericDistributedQueue.testPeekElements-seed#[68590BA2D6E9217E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testPeekElements
   [junit4]   2> 2542812 INFO  (TEST-TestGenericDistributedQueue.testPeekElements-seed#[68590BA2D6E9217E]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testPeekElements
   [junit4]   2> 2542815 INFO  (TEST-TestGenericDistributedQueue.testDistributedQueue-seed#[68590BA2D6E9217E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDistributedQueue
   [junit4]   2> 2544832 INFO  (TEST-TestGenericDistributedQueue.testDistributedQueue-seed#[68590BA2D6E9217E]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDistributedQueue
   [junit4]   2> 2544835 INFO  (TEST-TestGenericDistributedQueue.testDistributedQueueBlocking-seed#[68590BA2D6E9217E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDistributedQueueBlocking
   [junit4]   2> 2547953 INFO  (TEST-TestGenericDistributedQueue.testDistributedQueueBlocking-seed#[68590BA2D6E9217E]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDistributedQueueBlocking
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGenericDistributedQueue -Dtests.method=testDistributedQueueBlocking -Dtests.seed=68590BA2D6E9217E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=zh-SG -Dtests.timezone=Atlantic/Stanley -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   3.16s J1 | TestGenericDistributedQueue.testDistributedQueueBlocking <<<
   [junit4]    > Throwable #1: java.util.concurrent.TimeoutException
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([68590BA2D6E9217E:2DF379DB92BB9D0A]:0)
   [junit4]    > 	at java.util.concurrent.FutureTask.get(FutureTask.java:205)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.testDistributedQueueBlocking(TestSimDistributedQueue.java:116)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestGenericDistributedQueue_68590BA2D6E9217E-001
   [junit4]   2> May 12, 2018 2:46:41 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> May 12, 2018 2:48:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue: 
   [junit4]   2>    1) Thread[id=11941, name=sdqtest--2789-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194)
   [junit4]   2>         at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167)
   [junit4]   2>         at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$1(TestSimDistributedQueue.java:107)
   [junit4]   2>         at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$799/1372856567.call(Unknown Source)
   [junit4]   2>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
   [junit4]   2>         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$10/1192810417.run(Unknown Source)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> May 12, 2018 2:48:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=11941, name=sdqtest--2789-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
   [junit4]   2> May 12, 2018 2:48:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=84, maxMBSortInHeap=5.091317176365977, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@255a138), locale=zh-SG, timezone=Atlantic/Stanley
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=216700232,total=502267904
   [junit4]   2> NOTE: All tests run in this JVM: [TestCloudDeleteByQuery, TestSQLHandlerNonCloud, UpdateLogTest, ZkControllerTest, CollectionsAPIDistributedZkTest, LeaderVoteWaitTimeoutTest, TestLuceneIndexBackCompat, BigEndianAscendingWordDeserializerTest, ParsingFieldUpdateProcessorsTest, ClassificationUpdateProcessorIntegrationTest, TestSimpleTrackingShardHandler, TestReRankQParserPlugin, SegmentsInfoRequestHandlerTest, TestStandardQParsers, TestSSLRandomization, TestDistribIDF, ResourceLoaderTest, TestDownShardTolerantSearch, CheckHdfsIndexTest, BJQParserTest, TestSearchPerf, TestDocTermOrdsUninvertLimit, ForceLeaderTest, TestDistributedGrouping, ManagedSchemaRoundRobinCloudTest, DocumentBuilderTest, OverseerTest, TestSolrCloudWithKerberosAlt, TestBM25SimilarityFactory, HdfsWriteToMultipleCollectionsTest, TestSchemalessBufferedUpdates, DeleteReplicaTest, CollectionReloadTest, TestElisionMultitermQuery, SignatureUpdateProcessorFactoryTest, TestMinMaxOnMultiValuedField, TestConfigSetsAPIExclusivity, BadComponentTest, TestInPlaceUpdatesDistrib, SpellCheckCollatorWithCollapseTest, TestCloudManagedSchema, TestSerializedLuceneMatchVersion, TestSolrJ, AlternateDirectoryTest, JavabinLoaderTest, SubstringBytesRefFilterTest, TestDeleteCollectionOnDownNodes, TestComputePlanAction, SolrJmxReporterCloudTest, TestStreamBody, AtomicUpdateProcessorFactoryTest, TestSurroundQueryParser, TestManagedResource, QueryParsingTest, ChaosMonkeySafeLeaderWithPullReplicasTest, ScheduledTriggerTest, ExplicitHLLTest, HLLUtilTest, TestLocalFSCloudBackupRestore, SuggesterTest, TestAddFieldRealTimeGet, TestClassicSimilarityFactory, TolerantUpdateProcessorTest, TemplateUpdateProcessorTest, PKIAuthenticationIntegrationTest, DistributedFacetPivotSmallAdvancedTest, ConnectionManagerTest, TestCursorMarkWithoutUniqueKey, TestMacros, ShardRoutingCustomTest, TestSolrCoreSnapshots, TestSystemCollAutoCreate, IndexSchemaTest, TestCollectionAPIs, TestHttpShardHandlerFactory, HdfsRecoveryZkTest, TestGenericDistributedQueue]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGenericDistributedQueue -Dtests.seed=68590BA2D6E9217E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=zh-SG -Dtests.timezone=Atlantic/Stanley -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestGenericDistributedQueue (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue: 
   [junit4]    >    1) Thread[id=11941, name=sdqtest--2789-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194)
   [junit4]    >         at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167)
   [junit4]    >         at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$1(TestSimDistributedQueue.java:107)
   [junit4]    >         at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$799/1372856567.call(Unknown Source)
   [junit4]    >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    >         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
   [junit4]    >         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$10/1192810417.run(Unknown Source)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([68590BA2D6E9217E]:0)
   [junit4] Completed [244/806 (1!)] on J1 in 87.18s, 4 tests, 2 errors <<< FAILURES!

[...truncated 258 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestLeaderElectionZkExpiry
   [junit4]   2> 3213494 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[68590BA2D6E9217E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderElectionZkExpiry_68590BA2D6E9217E-001/init-core-data-001
   [junit4]   2> 3213494 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[68590BA2D6E9217E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 3213496 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[68590BA2D6E9217E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 3213515 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderElectionWithZkExpiry
   [junit4]   2> 3213604 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@42304c62, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 3214066 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42304c62
   [junit4]   2> 3214250 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42304c62
   [junit4]   2> 3214250 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42304c62
   [junit4]   2> 3214251 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderElectionZkExpiry_68590BA2D6E9217E-001/testLeaderElectionWithZkExpiry-solr-001
   [junit4]   2> 3214286 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3214305 INFO  (Thread-4178) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3214305 INFO  (Thread-4178) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3214378 ERROR (Thread-4178) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3214423 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [    ] o.a.s.c.ZkTestServer start zk server on port:42578
   [junit4]   2> 3214504 INFO  (zkConnectionManagerCallback-3392-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3214638 INFO  (zkConnectionManagerCallback-3394-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3214712 INFO  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3215063 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3215076 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
   [junit4]   2> 3215077 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer (id=72610145639333890-dummy.host.com:8984_solr-n_0000000000) starting
   [junit4]   2> 3215214 INFO  (zkConnectionManagerCallback-3403-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3215376 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [n:dummy.host.com:8984_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42578/solr ready
   [junit4]   2> 3215441 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 3215567 INFO  (zkCallback-3402-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3215573 INFO  (zkCallback-3402-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 3215770 WARN  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 3215774 WARN  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 3216777 WARN  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]-SendThread(127.0.0.1:42578)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x101f68af1ec0002 has expired
   [junit4]   2> 3216789 WARN  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 3216789 WARN  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 3216790 INFO  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72610145639333890-dummy.host.com:8984_solr-n_0000000000) closing
   [junit4]   2> 3216894 INFO  (OverseerStateUpdate-72610145639333890-dummy.host.com:8984_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : dummy.host.com:8984_solr
   [junit4]   2> 3216911 ERROR (OverseerExitThread) [    ] o.a.s.c.Overseer could not read the data
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:306) ~[java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 3217201 ERROR (OverseerAutoScalingTriggerThread-72610145639333890-dummy.host.com:8984_solr-n_0000000000) [    ] o.a.s.c.a.ScheduledTriggers Failed to add trigger .auto_add_replicas
   [junit4]   2> org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/events
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:124) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:114) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:110) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueueFactory.makeQueue(ZkDistributedQueueFactory.java:36) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:49) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.<init>(ScheduledTriggers.java:527) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:206) [java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:234) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/events
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1105) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:304) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:304) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:483) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:97) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:122) ~[java/:?]
   [junit4]   2> 	... 8 more
   [junit4]   2> 3217205 WARN  (zkCallback-3395-thread-1) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3217221 WARN  (OverseerAutoScalingTriggerThread-72610145639333890-dummy.host.com:8984_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed, exiting.
   [junit4]   2> 3217240 WARN  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 3217275 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3217275 INFO  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 3217275 INFO  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 3217418 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 3217418 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 3218704 WARN  (zkConnectionManagerCallback-3396-thread-1-SendThread(127.0.0.1:42578)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x101f68af1ec0004 has expired
   [junit4]   2> 3218704 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 3218704 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 3218704 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 3218744 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3218744 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 3218744 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 3218882 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 3218882 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 3220895 WARN  (zkConnectionManagerCallback-3396-thread-1-SendThread(127.0.0.1:42578)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x101f68af1ec0005 has expired
   [junit4]   2> 3220895 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 3220895 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 3220895 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 3220931 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3220931 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 3220931 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 3221004 ERROR (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.ZkStateReader Error reading cluster properties from zookeeper
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterprops.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.loadClusterProperties(ZkStateReader.java:989) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:446) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:332) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:167) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:267) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 3221077 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 3221077 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 3222906 WARN  (zkConnectionManagerCallback-3396-thread-1-SendThread(127.0.0.1:42578)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x101f68af1ec0006 has expired
   [junit4]   2> 3222909 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 3222909 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 3222909 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 3222930 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3222930 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 3222930 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 3223016 ERROR (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1535)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:328)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:761)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:447)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:332)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:167)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:267)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 
   [junit4]   2> 3223016 ERROR (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:417)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:167)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:267)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1535)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:328)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:761)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:447)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:332)
   [junit4]   2> 	... 10 more
   [junit4]   2> 
   [junit4]   2> 3223016 WARN  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 3223017 INFO  (zkConnectionManagerCallback-3396-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 3223056 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 3223056 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 3224769 WARN  (zkConnectionManagerCallback-3396-thread-1-SendThread(127.0.0.1:42578)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x101f68af1ec0007 has expired
   [junit4]   2> 3224769 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 3224769 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 3224769 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 3224814 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3224814 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 3224814 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 3224887 ERROR (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1105)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:316)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:440)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:332)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:167)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
   [junit4]   2> 
   [junit4]   2> 3224887 ERROR (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:417)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:167)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1105)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:316)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:440)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:332)
   [junit4]   2> 	... 5 more
   [junit4]   2> 
   [junit4]   2> 3224887 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 3224887 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 3224939 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 3224940 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 3225605 INFO  (zkConnectionManagerCallback-3405-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3226291 WARN  (zkConnectionManagerCallback-3396-thread-1-SendThread(127.0.0.1:42578)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x101f68af1ec0008 has expired
   [junit4]   2> 3226292 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791dd0a2 name: ZooKeeperConnection Watcher:127.0.0.1:42578/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 3226292 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 3226292 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 3226344 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3226344 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 3226344 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 3226589 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.LeaderElector Deleting duplicate registration: /overseer_elect/election/72610145639333898-dummy.host.com:8984_solr-n_0000000001
   [junit4]   2> 3226590 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
   [junit4]   2> 3226591 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[68590BA2D6E9217E]) [n:dummy.host.com:8984_solr    ] o.a.s.c.TestLeaderElectionZkExpiry Time=8431280565810053 Overseer leader is = dummy.host.com:8984_solr
   [junit4]   2> 3226605 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.Overseer Overseer (id=72610145639333898-dummy.host.com:8984_solr-n_0000000002) starting
   [junit4]   2> 3226612 WARN  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.LeaderElector Our node is no longer in line to be leader
   [junit4]   2> 3226612 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 3226624 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 3226625 ERROR (OverseerAutoScalingTriggerThread-72610145639333898-dummy.host.com:8984_solr-n_0000000002) [    ] o.a.s.c.a.OverseerTriggerThread A ZK error has occurred
   [junit4]   2> java.io.IOException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
   [junit4]   2> 	at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getAutoScalingConfig(ZkDistribStateManager.java:183) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.client.solrj.cloud.DistribStateManager.getAutoScalingConfig(DistribStateManager.java:83) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:131) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getAutoScalingConfig(ZkDistribStateManager.java:176) ~[java/:?]
   [junit4]   2> 	... 3 more
   [junit4]   2> 3226626 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 3226626 INFO  (zkConnectionManagerCallback-3396-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkC

[...truncated too long message...]

BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null577256664
     [copy] Copying 40 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null577256664
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null577256664

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1485632907
     [copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1485632907
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1485632907

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 99ms :: artifacts dl 18ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 315 minutes 21 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

Mime
View raw message