lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-master-Linux (64bit/jdk-9) - Build # 20606 - Still Failing!
Date Wed, 04 Oct 2017 14:36:06 GMT
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/20606/
Java: 64bit/jdk-9 -XX:-UseCompressedOops -XX:+UseParallelGC --illegal-access=deny

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:     1) Thread[id=14903, name=searcherExecutor-6469-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)         at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 
   1) Thread[id=14903, name=searcherExecutor-6469-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
        at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
	at __randomizedtesting.SeedInfo.seed([859FC622C520A965]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=14903, name=searcherExecutor-6469-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)         at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=14903, name=searcherExecutor-6469-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
        at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
	at __randomizedtesting.SeedInfo.seed([859FC622C520A965]:0)


FAILED:  org.apache.solr.core.TestLazyCores.testNoCommit

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([859FC622C520A965:5AFF67F30E07CAC0]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:884)
	at org.apache.solr.core.TestLazyCores.check10(TestLazyCores.java:847)
	at org.apache.solr.core.TestLazyCores.testNoCommit(TestLazyCores.java:829)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound='10']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int><lst name="params"><str name="q">*:*</str></lst></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=*:*
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:877)
	... 40 more


FAILED:  org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry

Error Message:
Captured an uncaught exception in thread: Thread[id=1200, name=OverseerAutoScalingTriggerThread-98770942456299522-dummy.host.com:8984_solr-n_0000000000, state=RUNNABLE, group=Overseer autoscaling triggers]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1200, name=OverseerAutoScalingTriggerThread-98770942456299522-dummy.host.com:8984_solr-n_0000000000, state=RUNNABLE, group=Overseer autoscaling triggers]
	at __randomizedtesting.SeedInfo.seed([859FC622C520A965:5340E778F33DBAFD]:0)
Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling
	at __randomizedtesting.SeedInfo.seed([859FC622C520A965]:0)
	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:107)
	at org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:44)
	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.<init>(ScheduledTriggers.java:398)
	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:149)
	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:220)
	at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:306)
	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:303)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:303)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:512)
	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:97)
	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:105)
	... 5 more




Build Log:
[...truncated 11588 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestLeaderElectionZkExpiry
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_859FC622C520A965-001/init-core-data-001
   [junit4]   2> 97175 WARN  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
   [junit4]   2> 97175 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 97176 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 97176 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 97176 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderElectionWithZkExpiry
   [junit4]   2> 97179 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 97205 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861
   [junit4]   2> 97210 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861
   [junit4]   2> 97210 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861
   [junit4]   2> 97210 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_859FC622C520A965-001/testLeaderElectionWithZkExpiry-solr-001
   [junit4]   2> 97211 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 97211 INFO  (Thread-305) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 97212 INFO  (Thread-305) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 97213 ERROR (Thread-305) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 97311 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.ZkTestServer start zk server on port:37151
   [junit4]   2> 97345 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 97345 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
   [junit4]   2> 97346 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.Overseer Overseer (id=98770942456299522-dummy.host.com:8984_solr-n_0000000000) starting
   [junit4]   2> 97347 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 97348 INFO  (zkCallback-204-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 97449 WARN  (zkCallback-204-thread-2) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 97450 WARN  (zkCallback-204-thread-2) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 98739 WARN  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]-SendThread(127.0.0.1:37151)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15ee7a61b3d0002 has expired
   [junit4]   2> 98739 WARN  (zkCallback-204-thread-6) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 98739 WARN  (zkCallback-204-thread-6) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 98739 INFO  (zkCallback-204-thread-6) [    ] o.a.s.c.Overseer Overseer (id=98770942456299522-dummy.host.com:8984_solr-n_0000000000) closing
   [junit4]   2> 98739 WARN  (OverseerCollectionConfigSetProcessor-98770942456299522-dummy.host.com:8984_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Overseer cannot talk to ZK
   [junit4]   2> 98740 WARN  (OverseerStateUpdate-98770942456299522-dummy.host.com:8984_solr-n_0000000000) [    ] o.a.s.c.Overseer Solr cannot talk to ZK, exiting Overseer main queue loop
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.fetchZkChildren(ZkDistributedQueue.java:313)
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.firstChild(ZkDistributedQueue.java:290)
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.firstElement(ZkDistributedQueue.java:406)
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.peek(ZkDistributedQueue.java:165)
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.peek(ZkDistributedQueue.java:142)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:185)
   [junit4]   2> 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 98741 INFO  (OverseerStateUpdate-98770942456299522-dummy.host.com:8984_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : dummy.host.com:8984_solr
   [junit4]   2> 98743 ERROR (OverseerExitThread) [    ] o.a.s.c.Overseer could not read the data
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:299)
   [junit4]   2> 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 98949 WARN  (zkCallback-204-thread-1) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> Oct 04, 2017 4:49:38 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[OverseerAutoScalingTriggerThread-98770942456299522-dummy.host.com:8984_solr-n_0000000000,5,Overseer autoscaling triggers]
   [junit4]   2> org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([859FC622C520A965]:0)
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:107)
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:44)
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.<init>(ScheduledTriggers.java:398)
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:149)
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:220)
   [junit4]   2> 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:306)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:303)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:303)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:512)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:97)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
   [junit4]   2> 	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:105)
   [junit4]   2> 	... 5 more
   [junit4]   2> 
   [junit4]   2> 98953 WARN  (zkCallback-204-thread-6) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 98955 INFO  (zkCallback-204-thread-6) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 98955 INFO  (zkCallback-204-thread-6) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 98957 INFO  (zkCallback-204-thread-6) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 99058 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 99058 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 100874 WARN  (zkCallback-204-thread-6-SendThread(127.0.0.1:37151)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15ee7a61b3d0003 has expired
   [junit4]   2> 100874 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 100874 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 100874 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 100876 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 100876 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 100974 WARN  (zkCallback-204-thread-6) [    ] o.a.s.c.ZkController Unable to read autoscaling.json
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:185)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:171)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:843)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:747)
   [junit4]   2> 	at java.base/java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4724)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:746)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:433)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:269)
   [junit4]   2> 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
   [junit4]   2> 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
   [junit4]   2> 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]   2> 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 100980 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 100980 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 102418 WARN  (zkCallback-204-thread-6-SendThread(127.0.0.1:37151)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15ee7a61b3d0004 has expired
   [junit4]   2> 102418 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 102418 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 102418 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 102420 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 102420 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 102527 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 102527 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 104198 WARN  (zkCallback-204-thread-6-SendThread(127.0.0.1:37151)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15ee7a61b3d0005 has expired
   [junit4]   2> 104198 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 104198 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 104199 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 104200 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 104200 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 104302 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 104302 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 105484 WARN  (zkCallback-204-thread-6-SendThread(127.0.0.1:37151)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15ee7a61b3d0006 has expired
   [junit4]   2> 105484 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 105484 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 105484 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 105486 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 105486 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 105596 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 105596 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 107170 WARN  (zkCallback-204-thread-6-SendThread(127.0.0.1:37151)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15ee7a61b3d0007 has expired
   [junit4]   2> 107170 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 107170 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 107170 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 107172 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 107172 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 107274 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 107274 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 108790 WARN  (zkCallback-204-thread-6-SendThread(127.0.0.1:37151)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15ee7a61b3d0008 has expired
   [junit4]   2> 108790 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67bfa73a name: ZooKeeperConnection Watcher:127.0.0.1:37151/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 108790 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 108790 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 108795 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 108795 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 110097 INFO  (zkCallback-204-thread-6) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
   [junit4]   2> 110098 INFO  (zkCallback-204-thread-6) [    ] o.a.s.c.Overseer Overseer (id=98770942456299530-dummy.host.com:8984_solr-n_0000000001) starting
   [junit4]   2> 110098 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.TestLeaderElectionZkExpiry Time=3043838648644151 Overseer leader is = dummy.host.com:8984_solr
   [junit4]   2> 110099 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.Overseer Overseer (id=98770942456299530-dummy.host.com:8984_solr-n_0000000001) closing
   [junit4]   2> 110099 INFO  (zkCallback-204-thread-6) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 110099 INFO  (OverseerStateUpdate-98770942456299530-dummy.host.com:8984_solr-n_0000000001) [    ] o.a.s.c.Overseer Overseer Loop exiting : dummy.host.com:8984_solr
   [junit4]   2> 110100 ERROR (OverseerCollectionConfigSetProcessor-98770942456299530-dummy.host.com:8984_solr-n_0000000001) [    ] o.a.s.c.OverseerTaskProcessor Unable to prioritize overseer 
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2> 	at java.base/java.lang.Object.wait(Native Method)
   [junit4]   2> 	at java.base/java.lang.Object.wait(Object.java:516)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1097)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:61)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:178)
   [junit4]   2> 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 110100 WARN  (OverseerExitThread) [    ] o.a.s.c.Overseer I'm exiting, but I'm still the leader
   [junit4]   2> 110101 ERROR (OverseerExitThread) [    ] o.a.s.c.Overseer Could not delete my leader node 
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:246)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:243)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:243)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:310)
   [junit4]   2> 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 110101 WARN  (OverseerExitThread) [    ] o.a.s.c.Overseer Unable to rejoinElection 
   [junit4]   2> org.apache.solr.common.SolrException: Unable to rejoin election
   [junit4]   2> 	at org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1924)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:324)
   [junit4]   2> 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect
   [junit4]   2> 	at org.apache.solr.cloud.OverseerElectionContext.<init>(ElectionContext.java:772)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerElectionContext.copy(ElectionContext.java:815)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:385)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1921)
   [junit4]   2> 	... 2 more
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerElectionContext.<init>(ElectionContext.java:770)
   [junit4]   2> 	... 5 more
   [junit4]   2> 110101 INFO  (zkCallback-204-thread-6) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 110101 INFO  (zkCallback-204-thread-6) [    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 111600 WARN  (zkCallback-204-thread-8) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 111601 WARN  (zkCallback-204-thread-5) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 111600 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ZkStateReader Error reading cluster properties from zookeeper
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterprops.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:306)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:303)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:303)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.loadClusterProperties(ZkStateReader.java:957)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:432)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 111601 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 111601 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 111601 WARN  (zkCallback-204-thread-1) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 111601 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.setup(LeaderElector.java:364)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:346)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 111601 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 111602 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.setup(LeaderElector.java:364)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:346)
   [junit4]   2> 	... 5 more
   [junit4]   2> 
   [junit4]   2> 111602 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 111601 WARN  (zkCallback-204-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 111602 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2> 	... 5 more
   [junit4]   2> 
   [junit4]   2> 111603 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 111602 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2> 	... 5 more
   [junit4]   2> 
   [junit4]   2> 111603 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 111603 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 111602 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2> 	... 5 more
   [junit4]   2> 
   [junit4]   2> 111604 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 111601 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:426)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 111601 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:716)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:433)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 111605 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:426)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2> 	... 5 more
   [junit4]   2> 
   [junit4]   2> 111605 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 111604 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 111605 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 111604 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1391816924
   [junit4]   2> 111604 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 111605 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 111603 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 111606 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@16f53d9: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@50dd9462
   [junit4]   2> 111605 ERROR (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:716)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:433)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2> 	... 5 more
   [junit4]   2> 
   [junit4]   2> 111606 WARN  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 111606 INFO  (zkCallback-204-thread-6-EventThread) [    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 111611 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 111611 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@19f7fd30: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@480a725a
   [junit4]   2> 111615 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 111615 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@29150bda: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@4e0f0083
   [junit4]   2> 111616 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 111616 ERROR (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 111617 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37151 37151
   [junit4]   2> 116651 INFO  (Thread-305) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37151 37151
   [junit4]   2> 116652 WARN  (Thread-305) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	6	/solr/security.json
   [junit4]   2> 	6	/solr/clusterprops.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	5	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 116652 INFO  (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[859FC622C520A965]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLeaderElectionWithZkExpiry
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLeaderElectionZkExpiry -Dtests.method=testLeaderElectionWithZkExpiry -Dtests.seed=859FC622C520A965 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en-ZW -Dtests.timezone=Africa/Khartoum -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   19.5s J0 | TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1200, name=OverseerAutoScalingTriggerThread-98770942456299522-dummy.host.com:8984_solr-n_0000000000, state=RUNNABLE, group=Overseer autoscaling triggers]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([859FC622C520A965:5340E778F33DBAFD]:0)
   [junit4]    > Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([859FC622C520A965]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:107)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:44)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.<init>(ScheduledTriggers.java:398)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:149)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:220)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]    > Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling
   [junit4]    > 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]    > 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]    > 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:306)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:303)
   [junit4]    > 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:303)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:512)
   [junit4]    > 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:97)
   [junit4]    > 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
   [junit4]    > 	at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:105)
   [junit4]    > 	... 5 more
   [junit4]   2> 116654 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 116654 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1391816924
   [junit4]   2> 116654 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 116654 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 116654 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 116654 INFO  (SUITE-TestLeaderElectionZkExpiry-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=183, maxMBSortInHeap=5.746101246625115, sim=RandomSimilarity(queryNorm=true): {}, locale=en-ZW, timezone=Africa/Khartoum
   [junit4]   2> NOTE: Linux 4.10.0-33-generic amd64/Oracle Corporation 9 (64-bit)/cpus=8,threads=1,free=146791784,total=516423680
   [junit4]   2> NOTE: All tests run in this JVM: [TestExclusionRuleCollectionAccess, DefaultValueUpdateProcessorTest, TestSolrConfigHandlerConcurrent, TestDefaultStatsCache, FieldMutatingUpdateProcessorTest, TestLegacyField, TestPivotHelperCode, TestPhraseSuggestions, HLLUtilTest, TestCollationField, TestDistributedStatsComponentCardinality, TestSSLRandomization, TestDeleteCollectionOnDownNodes, TestFastOutputStream, TestLeaderElectionZkExpiry]
   [junit4] Completed [39/745 (1!)] on J0 in 19.49s, 1 test, 1 error <<< FAILURES!

[...truncated 1268 lines...]
   [junit4] Suite: org.apache.solr.core.TestLazyCores
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/init-core-data-001
   [junit4]   2> 1863082 WARN  (SUITE-TestLazyCores-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=39 numCloses=39
   [junit4]   2> 1863082 INFO  (SUITE-TestLazyCores-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1863083 INFO  (SUITE-TestLazyCores-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 1863083 INFO  (SUITE-TestLazyCores-seed#[859FC622C520A965]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1863084 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCachingLimit
   [junit4]   2> 1863103 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/solr.xml
   [junit4]   2> 1863104 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1863105 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1863105 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1863128 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861
   [junit4]   2> 1863135 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861
   [junit4]   2> 1863135 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861
   [junit4]   2> 1863137 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1863141 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1863141 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1863146 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.s.IndexSchema [collection2] Schema name=tiny
   [junit4]   2> 1863146 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=tiny
   [junit4]   2> 1863148 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 1863148 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1863148 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/./collection1, trusted=true
   [junit4]   2> 1863148 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 1863148 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection2' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/./collection2, trusted=true
   [junit4]   2> 1863161 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861
   [junit4]   2> 1863161 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection2' (registry 'solr.core.collection2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861
   [junit4]   2> 1863162 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1863162 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1863162 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/./collection1/data/]
   [junit4]   2> 1863162 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.c.SolrCore [[collection2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/collection2], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/./collection2/data/]
   [junit4]   2> 1863163 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.s.IndexSchema [collection5] Schema name=tiny
   [junit4]   2> 1863163 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4ed3e368
   [junit4]   2> 1863163 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@d3ab845
   [junit4]   2> 1863164 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 1863164 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection5' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/./collection5, trusted=true
   [junit4]   2> 1863164 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection5' (registry 'solr.core.collection5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8214861
   [junit4]   2> 1863164 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1863164 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.c.SolrCore [[collection5] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/collection5], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/./collection5/data/]
   [junit4]   2> 1863165 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7139f498
   [junit4]   2> 1863281 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1863281 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1863282 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1863282 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@113f74e4[collection2] main]
   [junit4]   2> 1863282 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/collection2/conf
   [junit4]   2> 1863283 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1863283 INFO  (coreLoadExecutor-6257-thread-2) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1863283 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1863283 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1863283 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@95ebe71[collection5] main]
   [junit4]   2> 1863284 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/collection5/conf
   [junit4]   2> 1863284 INFO  (coreLoadExecutor-6257-thread-3) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1863284 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1863284 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1863284 INFO  (searcherExecutor-6260-thread-1) [    ] o.a.s.c.SolrCore [collection5] Registered new searcher Searcher@95ebe71[collection5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1863285 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1863285 INFO  (searcherExecutor-6259-thread-1) [    ] o.a.s.c.SolrCore [collection2] Registered new searcher Searcher@113f74e4[collection2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1863285 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@7908102[collection1] main]
   [junit4]   2> 1863285 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/collection1/conf
   [junit4]   2> 1863285 INFO  (coreLoadExecutor-6257-thread-1) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1863286 INFO  (searcherExecutor-6258-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7908102[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1863289 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1863300 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.s.IndexSchema [collection3] Schema name=tiny
   [junit4]   2> 1863302 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 1863302 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection3' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_859FC622C520A965-001/tempDir-001/./collection3, trusted=true
   [junit4]   2> 1863302 INFO  (TEST-TestLazyCores.testCachingLimit-seed#[859FC622C520A965]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection3' 

[...truncated too long message...]

junit4]   2>    1) Thread[id=14903, name=searcherExecutor-6469-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]   2>         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
   [junit4]   2>         at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
   [junit4]   2>         at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
   [junit4]   2>         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
   [junit4]   2>         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]   2>         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]   2>         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> Oct 04, 2017 2:19:35 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=14903, name=searcherExecutor-6469-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]   2> Oct 04, 2017 2:19:38 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=14903, name=searcherExecutor-6469-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]   2>         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
   [junit4]   2>         at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
   [junit4]   2>         at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
   [junit4]   2>         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
   [junit4]   2>         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]   2>         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]   2>         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=en-NA, timezone=America/Chicago
   [junit4]   2> NOTE: Linux 4.10.0-33-generic amd64/Oracle Corporation 9 (64-bit)/cpus=8,threads=2,free=291613752,total=531628032
   [junit4]   2> NOTE: All tests run in this JVM: [TestExclusionRuleCollectionAccess, DefaultValueUpdateProcessorTest, TestSolrConfigHandlerConcurrent, TestDefaultStatsCache, FieldMutatingUpdateProcessorTest, TestLegacyField, TestPivotHelperCode, TestPhraseSuggestions, HLLUtilTest, TestCollationField, TestDistributedStatsComponentCardinality, TestSSLRandomization, TestDeleteCollectionOnDownNodes, TestFastOutputStream, TestLeaderElectionZkExpiry, TestSubQueryTransformer, LeaderFailoverAfterPartitionTest, TestConfigSetProperties, TestFieldTypeCollectionResource, TestCSVLoader, TestFieldCacheWithThreads, ConjunctionSolrSpellCheckerTest, MultiThreadedOCPTest, SpellingQueryConverterTest, TestWordDelimiterFilterFactory, ShardSplitTest, TestLegacyTerms, TestInitQParser, TestComplexPhraseQParserPlugin, TestMacros, TestPushWriter, HdfsUnloadDistributedZkTest, TestIndexingPerformance, OutputWriterTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestEmbeddedSolrServerSchemaAPI, TestSolrCloudWithKerberosAlt, SimpleMLTQParserTest, TestLockTree, ExternalFileFieldSortTest, ExplicitHLLTest, TestReplicationHandlerBackup, TestExactStatsCache, BlockJoinFacetRandomTest, DistributedSpellCheckComponentTest, TestGeoJSONResponseWriter, ChaosMonkeyNothingIsSafeTest, SolrJmxReporterTest, SolrCoreTest, PeerSyncWithIndexFingerprintCachingTest, TestSolrCoreSnapshots, TestCloudSchemaless, CreateCollectionCleanupTest, HighlighterMaxOffsetTest, TestSizeLimitedDistributedMap, TestQueryWrapperFilter, ComputePlanActionTest, CursorPagingTest, TestNumericRangeQuery64, TestCustomStream, ConfigureRecoveryStrategyTest, XmlUpdateRequestHandlerTest, TestConfigSetsAPIExclusivity, UUIDFieldTest, SolrIndexSplitterTest, TestStandardQParsers, TestClusterStateMutator, HealthCheckHandlerTest, TestCSVResponseWriter, TestStressLucene, SolrIndexConfigTest, TriLevelCompositeIdRoutingTest, TestBadConfig, SegmentsInfoRequestHandlerTest, TestHighFrequencyDictionaryFactory, BufferStoreTest, JvmMetricsTest, UninvertDocValuesMergePolicyTest, TestStressInPlaceUpdates, TestTestInjection, TestSearcherReuse, BlobRepositoryCloudTest, MetricsHandlerTest, DeleteShardTest, SolrCLIZkUtilsTest, OverriddenZkACLAndCredentialsProvidersTest, DistributedQueryElevationComponentTest, OpenExchangeRatesOrgProviderTest, DistributedVersionInfoTest, DistributedQueryComponentOptimizationTest, TestIntervalFaceting, TestConfig, CloudExitableDirectoryReaderTest, TestLuceneMatchVersion, TestCrossCoreJoin, TestLuceneIndexBackCompat, TestPHPSerializedResponseWriter, TestSchemaResource, TestCloudNestedDocsSort, WordBreakSolrSpellCheckerTest, TestRandomFlRTGCloud, SolrMetricManagerTest, TestNonDefinedSimilarityFactory, TestInfoStreamLogging, SolrShardReporterTest, TestBackupRepositoryFactory, TestBulkSchemaAPI, DeleteLastCustomShardedReplicaTest, ActionThrottleTest, SuggesterWFSTTest, TestPKIAuthenticationPlugin, SyncSliceTest, TestInPlaceUpdatesStandalone, TemplateUpdateProcessorTest, AnalyticsQueryTest, PeerSyncTest, CloneFieldUpdateProcessorFactoryTest, TestDocTermOrds, OverseerRolesTest, TestCloudRecovery, DistributedFacetPivotLargeTest, TestExtendedDismaxParser, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, TestReplicationHandler, ClusterStateUpdateTest, TestRandomFaceting, TestRandomDVFaceting, ZkCLITest, TestDistributedGrouping, TestFaceting, TestHashPartitioner, TestRealTimeGet, TestJoin, DistributedTermsComponentTest, TestCoreContainer, TestSolr4Spatial, TestGroupingSearch, QueryElevationComponentTest, ConvertedLegacyTest, TestSort, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLazyCores -Dtests.seed=859FC622C520A965 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en-NA -Dtests.timezone=America/Chicago -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J0 | TestLazyCores (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 
   [junit4]    >    1) Thread[id=14903, name=searcherExecutor-6469-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]    >         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
   [junit4]    >         at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]    >         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([859FC622C520A965]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=14903, name=searcherExecutor-6469-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]    >         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
   [junit4]    >         at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]    >         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([859FC622C520A965]:0)
   [junit4] Completed [405/745 (2!)] on J0 in 36.34s, 9 tests, 3 errors <<< FAILURES!

[...truncated 40763 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:826: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:706: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:693: Source checkout is dirty (unversioned/missing files) after running tests!!! Offending files:
* lucene/licenses/morfologik-ukrainian-search-3.7.5.jar.sha1

Total time: 64 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

Mime
View raw message