lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-6.x-Linux (64bit/jdk1.8.0_112) - Build # 2411 - Unstable!
Date Tue, 13 Dec 2016 14:54:13 GMT
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2411/
Java: 64bit/jdk1.8.0_112 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
expected:<3> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<3> but was:<2>
	at __randomizedtesting.SeedInfo.seed([99180D0FE7082967:D16D79BBE13B06F2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:516)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11567 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/init-core-data-001
   [junit4]   2> 691207 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99180D0FE7082967]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 691208 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99180D0FE7082967]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001
   [junit4]   2> 691208 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99180D0FE7082967]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 691208 INFO  (Thread-1403) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 691208 INFO  (Thread-1403) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 691308 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99180D0FE7082967]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:45329
   [junit4]   2> 691313 INFO  (jetty-launcher-25697-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 691313 INFO  (jetty-launcher-25697-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 691313 INFO  (jetty-launcher-25697-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 691314 INFO  (jetty-launcher-25697-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d9f3927{/solr,null,AVAILABLE}
   [junit4]   2> 691314 INFO  (jetty-launcher-25697-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 691314 INFO  (jetty-launcher-25697-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@71ba8ecb{/solr,null,AVAILABLE}
   [junit4]   2> 691314 INFO  (jetty-launcher-25697-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1e1f84c6{/solr,null,AVAILABLE}
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@15fa9e82{SSL,[ssl, http/1.1]}{127.0.0.1:37889}
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@520f1bfb{SSL,[ssl, http/1.1]}{127.0.0.1:33604}
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-3) [    ] o.e.j.s.Server Started @693398ms
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-2) [    ] o.e.j.s.Server Started @693398ms
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37889}
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33604}
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@42f6453f{SSL,[ssl, http/1.1]}{127.0.0.1:44207}
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@28190ade{/solr,null,AVAILABLE}
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-1) [    ] o.e.j.s.Server Started @693398ms
   [junit4]   2> 691316 ERROR (jetty-launcher-25697-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44207}
   [junit4]   2> 691316 INFO  (jetty-launcher-25697-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 6.4.0
   [junit4]   2> 691316 ERROR (jetty-launcher-25697-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 691317 ERROR (jetty-launcher-25697-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 6.4.0
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 6.4.0
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-12-13T14:19:58.441Z
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-12-13T14:19:58.441Z
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@754b4d50{SSL,[ssl, http/1.1]}{127.0.0.1:42642}
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-12-13T14:19:58.441Z
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-4) [    ] o.e.j.s.Server Started @693399ms
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42642}
   [junit4]   2> 691317 ERROR (jetty-launcher-25697-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 6.4.0
   [junit4]   2> 691317 INFO  (jetty-launcher-25697-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 691318 INFO  (jetty-launcher-25697-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 691318 INFO  (jetty-launcher-25697-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-12-13T14:19:58.442Z
   [junit4]   2> 691318 INFO  (jetty-launcher-25697-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 691318 INFO  (jetty-launcher-25697-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 691318 INFO  (jetty-launcher-25697-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 691319 INFO  (jetty-launcher-25697-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 691326 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x158f88de3fd0001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 691326 INFO  (jetty-launcher-25697-thread-3) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 691326 INFO  (jetty-launcher-25697-thread-1) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 691327 INFO  (jetty-launcher-25697-thread-4) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 691327 INFO  (jetty-launcher-25697-thread-2) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 691327 INFO  (jetty-launcher-25697-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45329/solr
   [junit4]   2> 691327 INFO  (jetty-launcher-25697-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45329/solr
   [junit4]   2> 691328 INFO  (jetty-launcher-25697-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45329/solr
   [junit4]   2> 691329 INFO  (jetty-launcher-25697-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45329/solr
   [junit4]   2> 691330 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x158f88de3fd0008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 691341 INFO  (jetty-launcher-25697-thread-1) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44207_solr
   [junit4]   2> 691341 INFO  (jetty-launcher-25697-thread-4) [n:127.0.0.1:42642_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42642_solr
   [junit4]   2> 691341 INFO  (jetty-launcher-25697-thread-2) [n:127.0.0.1:33604_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33604_solr
   [junit4]   2> 691342 INFO  (jetty-launcher-25697-thread-1) [n:127.0.0.1:44207_solr    ] o.a.s.c.Overseer Overseer (id=97100680287551498-127.0.0.1:44207_solr-n_0000000000) starting
   [junit4]   2> 691343 INFO  (zkCallback-25718-thread-1-processing-n:127.0.0.1:33604_solr) [n:127.0.0.1:33604_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 691343 INFO  (zkCallback-25715-thread-1-processing-n:127.0.0.1:42642_solr) [n:127.0.0.1:42642_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 691344 INFO  (zkCallback-25716-thread-1-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 691344 INFO  (zkCallback-25717-thread-1-processing-n:127.0.0.1:37889_solr) [n:127.0.0.1:37889_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 691345 INFO  (jetty-launcher-25697-thread-3) [n:127.0.0.1:37889_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37889_solr
   [junit4]   2> 691345 INFO  (zkCallback-25717-thread-1-processing-n:127.0.0.1:37889_solr) [n:127.0.0.1:37889_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 691345 INFO  (zkCallback-25715-thread-1-processing-n:127.0.0.1:42642_solr) [n:127.0.0.1:42642_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 691345 INFO  (zkCallback-25716-thread-1-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 691353 INFO  (zkCallback-25718-thread-1-processing-n:127.0.0.1:33604_solr) [n:127.0.0.1:33604_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 691355 INFO  (jetty-launcher-25697-thread-1) [n:127.0.0.1:44207_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44207_solr
   [junit4]   2> 691356 INFO  (zkCallback-25718-thread-1-processing-n:127.0.0.1:33604_solr) [n:127.0.0.1:33604_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 691356 INFO  (zkCallback-25716-thread-1-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 691356 INFO  (zkCallback-25715-thread-1-processing-n:127.0.0.1:42642_solr) [n:127.0.0.1:42642_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 691360 INFO  (zkCallback-25717-thread-1-processing-n:127.0.0.1:37889_solr) [n:127.0.0.1:37889_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 691414 INFO  (jetty-launcher-25697-thread-2) [n:127.0.0.1:33604_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node2/.
   [junit4]   2> 691427 INFO  (jetty-launcher-25697-thread-3) [n:127.0.0.1:37889_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node3/.
   [junit4]   2> 691453 INFO  (jetty-launcher-25697-thread-1) [n:127.0.0.1:44207_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node1/.
   [junit4]   2> 691460 INFO  (jetty-launcher-25697-thread-4) [n:127.0.0.1:42642_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node4/.
   [junit4]   2> 691463 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99180D0FE7082967]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 691463 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99180D0FE7082967]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45329/solr ready
   [junit4]   2> 691485 INFO  (TEST-CollectionsAPIDistributedZkTest.testMaxNodesPerShard-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMaxNodesPerShard
   [junit4]   2> 691486 INFO  (TEST-CollectionsAPIDistributedZkTest.testMaxNodesPerShard-seed#[99180D0FE7082967]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 691542 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=oversharded&action=CREATE&numShards=3&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 691543 INFO  (OverseerThreadFactory-7318-thread-1-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.CreateCollectionCmd Create collection oversharded
   [junit4]   2> 691543 ERROR (OverseerThreadFactory-7318-thread-1-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerCollectionMessageHandler Collection: oversharded operation: create failed:org.apache.solr.common.SolrException: Cannot create collection oversharded. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 3 and value of replicationFactor is 2. This requires 6 shards to be created (higher than the allowed number)
   [junit4]   2> 	at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:155)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 691546 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=oversharded&action=CREATE&numShards=3&wt=javabin&version=2} status=400 QTime=4
   [junit4]   2> 691546 INFO  (TEST-CollectionsAPIDistributedZkTest.testMaxNodesPerShard-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testMaxNodesPerShard
   [junit4]   2> 691565 INFO  (TEST-CollectionsAPIDistributedZkTest.testNoConfigSetExist-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNoConfigSetExist
   [junit4]   2> 691566 INFO  (TEST-CollectionsAPIDistributedZkTest.testNoConfigSetExist-seed#[99180D0FE7082967]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 691570 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-002&collection.configName=conf123&name=corewithnocollection3&action=CREATE&numShards=1&collection=&wt=javabin&version=2
   [junit4]   2> 691571 INFO  (OverseerStateUpdate-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 692573 ERROR (qtp1321518926-95886) [n:127.0.0.1:37889_solr c:corewithnocollection3   x:corewithnocollection3] o.a.s.c.c.ZkStateReader Specified config=[conf123] does not exist in ZooKeeper at location=[/configs/conf123]
   [junit4]   2> 692573 ERROR (qtp1321518926-95886) [n:127.0.0.1:37889_solr c:corewithnocollection3   x:corewithnocollection3] o.a.s.c.CoreContainer Error creating core [corewithnocollection3]: Specified config does not exist in ZooKeeper: conf123
   [junit4]   2> org.apache.solr.common.cloud.ZooKeeperException: Specified config does not exist in ZooKeeper: conf123
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:189)
   [junit4]   2> 	at org.apache.solr.cloud.CloudConfigSetService.createCoreResourceLoader(CloudConfigSetService.java:54)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:74)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:840)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
   [junit4]   2> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:152)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 692574 ERROR (qtp1321518926-95886) [n:127.0.0.1:37889_solr c:corewithnocollection3   x:corewithnocollection3] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error CREATEing SolrCore 'corewithnocollection3': Unable to create core [corewithnocollection3] Caused by: Specified config does not exist in ZooKeeper: conf123
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:813)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
   [junit4]   2> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:152)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unable to create core [corewithnocollection3]
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:855)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
   [junit4]   2> 	... 36 more
   [junit4]   2> Caused by: org.apache.solr.common.cloud.ZooKeeperException: Specified config does not exist in ZooKeeper: conf123
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:189)
   [junit4]   2> 	at org.apache.solr.cloud.CloudConfigSetService.createCoreResourceLoader(CloudConfigSetService.java:54)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:74)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:840)
   [junit4]   2> 	... 37 more
   [junit4]   2> 
   [junit4]   2> 692574 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr c:corewithnocollection3   x:corewithnocollection3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={dataDir=/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-002&collection.configName=conf123&name=corewithnocollection3&action=CREATE&numShards=1&collection=&wt=javabin&version=2} status=400 QTime=1005
   [junit4]   2> 692778 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 692778 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 692778 INFO  (TEST-CollectionsAPIDistributedZkTest.testNoConfigSetExist-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNoConfigSetExist
   [junit4]   2> 692796 INFO  (TEST-CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Starting deletePartiallyCreatedCollection
   [junit4]   2> 692796 INFO  (TEST-CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection-seed#[99180D0FE7082967]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 692798 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=corewithnocollection3&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 692800 INFO  (OverseerThreadFactory-7318-thread-2-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 692800 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 693502 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=corewithnocollection3&action=DELETE&wt=javabin&version=2} status=0 QTime=703
   [junit4]   2> 693504 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-003&collection.configName=conf&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&wt=javabin&version=2
   [junit4]   2> 693505 INFO  (OverseerStateUpdate-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 694512 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 694519 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema [halfdeletedcollection_shard1_replica1] Schema name=minimal
   [junit4]   2> 694521 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 694522 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection_shard1_replica1' using configuration from collection halfdeletedcollection
   [junit4]   2> 694522 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [[halfdeletedcollection_shard1_replica1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node3/halfdeletedcollection_shard1_replica1], dataDir=[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-003/]
   [junit4]   2> 694528 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 694528 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 694529 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 694529 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 694529 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1d91443[halfdeletedcollection_shard1_replica1] main]
   [junit4]   2> 694530 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 694530 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 694530 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 694530 INFO  (searcherExecutor-7327-thread-1-processing-n:127.0.0.1:37889_solr x:halfdeletedcollection_shard1_replica1 c:halfdeletedcollection) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1] Registered new searcher Searcher@1d91443[halfdeletedcollection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 694530 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1553610888083144704
   [junit4]   2> 694534 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 694534 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 694534 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37889/solr/halfdeletedcollection_shard1_replica1/
   [junit4]   2> 694534 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 694534 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:37889/solr/halfdeletedcollection_shard1_replica1/ has no replicas
   [junit4]   2> 694536 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37889/solr/halfdeletedcollection_shard1_replica1/ shard2
   [junit4]   2> 694687 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 694688 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={dataDir=/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-003&collection.configName=conf&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&wt=javabin&version=2} status=0 QTime=1184
   [junit4]   2> 694690 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=halfdeletedcollection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 694691 INFO  (OverseerThreadFactory-7318-thread-3-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 694691 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 694695 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5f0a7a6b
   [junit4]   2> 694697 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfdeletedcollection_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 695400 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=halfdeletedcollection&action=DELETE&wt=javabin&version=2} status=0 QTime=709
   [junit4]   2> 695402 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 695402 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 695403 INFO  (qtp1321518926-95874) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=halfdeletedcollection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 695405 INFO  (OverseerThreadFactory-7318-thread-4-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.CreateCollectionCmd Create collection halfdeletedcollection
   [junit4]   2> 695405 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 695610 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard2_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard2&wt=javabin&version=2
   [junit4]   2> 695661 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard1&wt=javabin&version=2
   [junit4]   2> 695763 INFO  (zkCallback-25718-thread-1-processing-n:127.0.0.1:33604_solr) [n:127.0.0.1:33604_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 695763 INFO  (zkCallback-25717-thread-1-processing-n:127.0.0.1:37889_solr) [n:127.0.0.1:37889_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 696617 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 696624 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.s.IndexSchema [halfdeletedcollection_shard2_replica1] Schema name=minimal
   [junit4]   2> 696642 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 696644 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection_shard2_replica1' using configuration from collection halfdeletedcollection
   [junit4]   2> 696644 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrCore [[halfdeletedcollection_shard2_replica1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node3/halfdeletedcollection_shard2_replica1], dataDir=[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node3/./halfdeletedcollection_shard2_replica1/data/]
   [junit4]   2> 696650 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 696650 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 696651 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 696651 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 696651 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2bba0683[halfdeletedcollection_shard2_replica1] main]
   [junit4]   2> 696652 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 696652 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 696652 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 696652 INFO  (searcherExecutor-7332-thread-1-processing-n:127.0.0.1:37889_solr x:halfdeletedcollection_shard2_replica1 s:shard2 c:halfdeletedcollection) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrCore [halfdeletedcollection_shard2_replica1] Registered new searcher Searcher@2bba0683[halfdeletedcollection_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 696652 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1553610890308222976
   [junit4]   2> 696656 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 696656 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 696656 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37889/solr/halfdeletedcollection_shard2_replica1/
   [junit4]   2> 696656 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 696656 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:37889/solr/halfdeletedcollection_shard2_replica1/ has no replicas
   [junit4]   2> 696657 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37889/solr/halfdeletedcollection_shard2_replica1/ shard2
   [junit4]   2> 696668 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 696674 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema [halfdeletedcollection_shard1_replica1] Schema name=minimal
   [junit4]   2> 696676 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 696677 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection_shard1_replica1' using configuration from collection halfdeletedcollection
   [junit4]   2> 696677 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [[halfdeletedcollection_shard1_replica1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node2/halfdeletedcollection_shard1_replica1], dataDir=[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node2/./halfdeletedcollection_shard1_replica1/data/]
   [junit4]   2> 696681 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 696681 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 696682 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 696682 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 696682 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@30d2fed1[halfdeletedcollection_shard1_replica1] main]
   [junit4]   2> 696683 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 696683 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 696683 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 696683 INFO  (searcherExecutor-7337-thread-1-processing-n:127.0.0.1:33604_solr x:halfdeletedcollection_shard1_replica1 s:shard1 c:halfdeletedcollection) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1] Registered new searcher Searcher@30d2fed1[halfdeletedcollection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 696683 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1553610890340728832
   [junit4]   2> 696687 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 696687 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 696687 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:33604/solr/halfdeletedcollection_shard1_replica1/
   [junit4]   2> 696687 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 696687 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:33604/solr/halfdeletedcollection_shard1_replica1/ has no replicas
   [junit4]   2> 696689 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:33604/solr/halfdeletedcollection_shard1_replica1/ shard1
   [junit4]   2> 696790 INFO  (zkCallback-25717-thread-1-processing-n:127.0.0.1:37889_solr) [n:127.0.0.1:37889_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 696790 INFO  (zkCallback-25718-thread-1-processing-n:127.0.0.1:33604_solr) [n:127.0.0.1:33604_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 696808 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 696809 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard2_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard2&wt=javabin&version=2} status=0 QTime=1199
   [junit4]   2> 696840 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 696841 INFO  (qtp1179644305-95867) [n:127.0.0.1:33604_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard1&wt=javabin&version=2} status=0 QTime=1179
   [junit4]   2> 696842 INFO  (qtp1321518926-95874) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 696942 INFO  (zkCallback-25718-thread-1-processing-n:127.0.0.1:33604_solr) [n:127.0.0.1:33604_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 696942 INFO  (zkCallback-25717-thread-1-processing-n:127.0.0.1:37889_solr) [n:127.0.0.1:37889_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 697406 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 697842 INFO  (qtp1321518926-95874) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=halfdeletedcollection&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2438
   [junit4]   2> 697844 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 697844 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 697844 INFO  (TEST-CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Ending deletePartiallyCreatedCollection
   [junit4]   2> 697862 INFO  (TEST-CollectionsAPIDistributedZkTest.testCreateNodeSet-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCreateNodeSet
   [junit4]   2> 697862 INFO  (TEST-CollectionsAPIDistributedZkTest.testCreateNodeSet-seed#[99180D0FE7082967]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 697864 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=halfdeletedcollection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 697865 INFO  (OverseerThreadFactory-7318-thread-5-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 697868 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr    ] o.a.s.c.SolrCore [halfdeletedcollection_shard2_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2109566d
   [junit4]   2> 697868 INFO  (qtp1179644305-95876) [n:127.0.0.1:33604_solr    ] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@18f634f6
   [junit4]   2> 697871 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfdeletedcollection_shard2_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 697871 INFO  (qtp1179644305-95876) [n:127.0.0.1:33604_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfdeletedcollection_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 698573 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=halfdeletedcollection&action=DELETE&wt=javabin&version=2} status=0 QTime=709
   [junit4]   2> 698575 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=nodeset_collection&action=CREATE&numShards=2&createNodeSet=https://127.0.0.1:44207/solr,https://127.0.0.1:44207/solr&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 698576 INFO  (OverseerThreadFactory-7318-thread-5-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.CreateCollectionCmd Create collection nodeset_collection
   [junit4]   2> 698577 WARN  (OverseerThreadFactory-7318-thread-5-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (nodeset_collection) without cores.
   [junit4]   2> 698577 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 698780 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 698780 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=nodeset_collection&action=CREATE&numShards=2&createNodeSet=https://127.0.0.1:44207/solr,https://127.0.0.1:44207/solr&wt=javabin&version=2} status=0 QTime=204
   [junit4]   2> 698780 INFO  (TEST-CollectionsAPIDistributedZkTest.testCreateNodeSet-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testCreateNodeSet
   [junit4]   2> 698799 INFO  (TEST-CollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
   [junit4]   2> 698800 INFO  (TEST-CollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[99180D0FE7082967]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 698801 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=nodeset_collection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 698802 INFO  (OverseerThreadFactory-7318-thread-5-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 698802 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 699504 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=nodeset_collection&action=DELETE&wt=javabin&version=2} status=0 QTime=703
   [junit4]   2> 699507 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 699508 INFO  (OverseerThreadFactory-7318-thread-5-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.CreateCollectionCmd Create collection collection
   [junit4]   2> 699508 WARN  (OverseerThreadFactory-7318-thread-5-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.CreateCollectionCmd Specified replicationFactor of 10 on collection collection is higher than or equal to the number of Solr instances currently live or live and part of your createNodeSet(4). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 699508 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000012 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 699508 ERROR (OverseerThreadFactory-7318-thread-5-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerCollectionMessageHandler Collection: collection operation: create failed:org.apache.solr.common.SolrException: Cannot create collection collection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 2 and value of replicationFactor is 10. This requires 20 shards to be created (higher than the allowed number)
   [junit4]   2> 	at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:155)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 699509 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2} status=400 QTime=2
   [junit4]   2> 699509 INFO  (TEST-CollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
   [junit4]   2> 699529 INFO  (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Starting deleteCollectionOnlyInZk
   [junit4]   2> 699530 INFO  (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[99180D0FE7082967]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 699532 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 699533 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000014 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 699534 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 699536 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 699536 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 699538 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=onlyinzk&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 699538 INFO  (OverseerThreadFactory-7318-thread-5-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.CreateCollectionCmd Create collection onlyinzk
   [junit4]   2> 699539 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000016 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 699743 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2
   [junit4]   2> 699793 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2
   [junit4]   2> 699895 INFO  (zkCallback-25718-thread-1-processing-n:127.0.0.1:33604_solr) [n:127.0.0.1:33604_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 699895 INFO  (zkCallback-25715-thread-1-processing-n:127.0.0.1:42642_solr) [n:127.0.0.1:42642_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 700753 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 700762 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.IndexSchema [onlyinzk_shard2_replica1] Schema name=minimal
   [junit4]   2> 700765 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 700766 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'onlyinzk_shard2_replica1' using configuration from collection onlyinzk
   [junit4]   2> 700766 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SolrCore [[onlyinzk_shard2_replica1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node2/onlyinzk_shard2_replica1], dataDir=[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node2/./onlyinzk_shard2_replica1/data/]
   [junit4]   2> 700775 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 700775 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 700776 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 700776 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 700777 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@539f2564[onlyinzk_shard2_replica1] main]
   [junit4]   2> 700777 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 700778 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 700778 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 700778 INFO  (searcherExecutor-7342-thread-1-processing-n:127.0.0.1:33604_solr x:onlyinzk_shard2_replica1 s:shard2 c:onlyinzk) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SolrCore [onlyinzk_shard2_replica1] Registered new searcher Searcher@539f2564[onlyinzk_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 700778 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1553610894634647552
   [junit4]   2> 700783 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 700783 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 700783 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:33604/solr/onlyinzk_shard2_replica1/
   [junit4]   2> 700783 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 700783 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:33604/solr/onlyinzk_shard2_replica1/ has no replicas
   [junit4]   2> 700785 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:33604/solr/onlyinzk_shard2_replica1/ shard2
   [junit4]   2> 700820 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 700826 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.IndexSchema [onlyinzk_shard1_replica1] Schema name=minimal
   [junit4]   2> 700829 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 700830 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'onlyinzk_shard1_replica1' using configuration from collection onlyinzk
   [junit4]   2> 700830 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SolrCore [[onlyinzk_shard1_replica1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node4/onlyinzk_shard1_replica1], dataDir=[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001/tempDir-001/node4/./onlyinzk_shard1_replica1/data/]
   [junit4]   2> 700835 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 700835 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 700836 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 700836 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 700836 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@cee843a[onlyinzk_shard1_replica1] main]
   [junit4]   2> 700836 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 700837 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 700837 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 700837 INFO  (searcherExecutor-7347-thread-1-processing-n:127.0.0.1:42642_solr x:onlyinzk_shard1_replica1 s:shard1 c:onlyinzk) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SolrCore [onlyinzk_shard1_replica1] Registered new searcher Searcher@cee843a[onlyinzk_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 700837 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1553610894696513536
   [junit4]   2> 700841 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 700841 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 700841 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:42642/solr/onlyinzk_shard1_replica1/
   [junit4]   2> 700841 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 700841 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:42642/solr/onlyinzk_shard1_replica1/ has no replicas
   [junit4]   2> 700843 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:42642/solr/onlyinzk_shard1_replica1/ shard1
   [junit4]   2> 700944 INFO  (zkCallback-25718-thread-1-processing-n:127.0.0.1:33604_solr) [n:127.0.0.1:33604_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 700944 INFO  (zkCallback-25715-thread-1-processing-n:127.0.0.1:42642_solr) [n:127.0.0.1:42642_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 700986 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 700987 INFO  (qtp1179644305-95877) [n:127.0.0.1:33604_solr c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2} status=0 QTime=1243
   [junit4]   2> 700993 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 700994 INFO  (qtp1707862002-95896) [n:127.0.0.1:42642_solr c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2} status=0 QTime=1201
   [junit4]   2> 700995 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 701095 INFO  (zkCallback-25718-thread-1-processing-n:127.0.0.1:33604_solr) [n:127.0.0.1:33604_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 701095 INFO  (zkCallback-25715-thread-1-processing-n:127.0.0.1:42642_solr) [n:127.0.0.1:42642_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 701539 INFO  (OverseerCollectionConfigSetProcessor-97100680287551498-127.0.0.1:44207_solr-n_0000000000) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000018 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 701996 INFO  (qtp1321518926-95890) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=onlyinzk&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2458
   [junit4]   2> 701998 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 701998 INFO  (qtp1321518926-95886) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 701998 INFO  (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Ending deleteCollectionOnlyInZk
   [junit4]   2> 702020 INFO  (TEST-CollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes-seed#[99180D0FE7082967]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCoresAreDistributedAcrossNodes
   [junit4]   2> 702020 INFO  (TEST-CollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes-seed#[99180D0FE7082967]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 702022 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 702023 INFO  (OverseerThreadFactory-7318-thread-5-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 702025 INFO  (qtp1707862002-95902) [n:127.0.0.1:42642_solr    ] o.a.s.c.SolrCore [onlyinzk_shard1_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@62e84b68
   [junit4]   2> 702025 INFO  (qtp1179644305-95880) [n:127.0.0.1:33604_solr    ] o.a.s.c.SolrCore [onlyinzk_shard2_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@72e348df
   [junit4]   2> 702028 INFO  (qtp1707862002-95902) [n:127.0.0.1:42642_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=onlyinzk_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 702028 INFO  (qtp1179644305-95880) [n:127.0.0.1:33604_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=onlyinzk_shard2_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 702731 INFO  (qtp1321518926-95887) [n:127.0.0.1:37889_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=708
   [junit4]   2> 702732 INFO  (qtp1321518926-95869) [n:127.0.0.1:37889_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :cr

[...truncated too long message...]

ler.java:365)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
   [junit4]   2> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:152)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 781822 INFO  (qtp1595662130-95875) [n:127.0.0.1:44207_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:37889_solr&onlyIfLeaderActive=true&core=addReplicaColl_shard1_replica2&coreNodeName=core_node7&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=400 QTime=14006
   [junit4]   2> 782539 WARN  (zkCallback-25715-thread-2-processing-n:127.0.0.1:42642_solr) [n:127.0.0.1:42642_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 782540 INFO  (jetty-launcher-25697-thread-8) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@28190ade{/solr,null,UNAVAILABLE}
   [junit4]   2> 782540 INFO  (jetty-launcher-25697-thread-5) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2d9f3927{/solr,null,UNAVAILABLE}
   [junit4]   2> 782541 WARN  (zkCallback-25717-thread-1-processing-n:127.0.0.1:37889_solr) [n:127.0.0.1:37889_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 782541 WARN  (zkCallback-25716-thread-8-processing-n:127.0.0.1:44207_solr) [n:127.0.0.1:44207_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 782542 INFO  (jetty-launcher-25697-thread-6) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@71ba8ecb{/solr,null,UNAVAILABLE}
   [junit4]   2> 782542 INFO  (jetty-launcher-25697-thread-7) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1e1f84c6{/solr,null,UNAVAILABLE}
   [junit4]   2> 782542 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99180D0FE7082967]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45329 45329
   [junit4]   2> 782556 INFO  (Thread-1440) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45329 45329
   [junit4]   2> 782557 WARN  (Thread-1440) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	10	/solr/configs/conf
   [junit4]   2> 	8	/solr/aliases.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	88	/solr/clusterstate.json
   [junit4]   2> 	8	/solr/clusterprops.json
   [junit4]   2> 	4	/solr/collections/addReplicaColl/state.json
   [junit4]   2> 	4	/solr/collections/reloaded_collection/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	222	/solr/collections
   [junit4]   2> 	41	/solr/overseer/queue
   [junit4]   2> 	8	/solr/live_nodes
   [junit4]   2> 	7	/solr/overseer/queue-work
   [junit4]   2> 	6	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> 782557 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99180D0FE7082967]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_99180D0FE7082967-001
   [junit4]   2> Dec 13, 2016 2:21:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene62, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=tr, timezone=Pacific/Kiritimati
   [junit4]   2> NOTE: Linux 4.4.0-53-generic amd64/Oracle Corporation 1.8.0_112 (64-bit)/cpus=12,threads=1,free=74295912,total=509673472
   [junit4]   2> NOTE: All tests run in this JVM: [TestDocBasedVersionConstraints, SolrCLIZkUtilsTest, TestFiltering, ConvertedLegacyTest, ConfigSetsAPITest, DocValuesNotIndexedTest, TestRestoreCore, TestCloudPivotFacet, TestClusterProperties, HLLUtilTest, TestInitParams, TestPartialUpdateDeduplication, SortSpecParsingTest, TestAuthenticationFramework, TestSchemaResource, HdfsDirectoryFactoryTest, DistributedFacetPivotLargeTest, SpellingQueryConverterTest, TestSlowCompositeReaderWrapper, BasicFunctionalityTest, TestDistributedGrouping, TestDynamicFieldResource, TestTrie, SuggestComponentContextFilterQueryTest, OverseerCollectionConfigSetProcessorTest, TestNamedUpdateProcessors, TestChildDocTransformer, TestSolr4Spatial2, TestStressUserVersions, TestUpdate, JavabinLoaderTest, DirectSolrSpellCheckerTest, TestIntervalFaceting, TestSolrCLIRunExample, TestStressLiveNodes, TlogReplayBufferedWhileIndexingTest, DistributedQueryComponentOptimizationTest, FileUtilsTest, TestFieldTypeCollectionResource, TestFastOutputStream, TestFieldCacheSortRandom, LeaderInitiatedRecoveryOnCommitTest, TestLeaderElectionZkExpiry, IndexSchemaRuntimeFieldTest, SolrIndexConfigTest, CollectionReloadTest, HdfsUnloadDistributedZkTest, TestMissingGroups, CoreAdminRequestStatusTest, TestQueryTypes, SolrIndexSplitterTest, SimpleMLTQParserTest, TestSearcherReuse, BlockJoinFacetSimpleTest, TermVectorComponentTest, OpenCloseCoreStressTest, JSONWriterTest, ShardRoutingCustomTest, TestLRUCache, CdcrUpdateLogTest, BJQParserTest, TestLockTree, TestCSVResponseWriter, UpdateRequestProcessorFactoryTest, TestCloudManagedSchema, BigEndianAscendingWordDeserializerTest, DistanceFunctionTest, TestAnalyzeInfixSuggestions, TestScoreJoinQPScore, WordBreakSolrSpellCheckerTest, CursorMarkTest, HdfsChaosMonkeySafeLeaderTest, BasicAuthIntegrationTest, ShardRoutingTest, TestMiniSolrCloudClusterSSL, DistributedFacetPivotSmallTest, ClusterStateUpdateTest, TestSubQueryTransformerDistrib, TestQueryWrapperFilter, TestRestManager, CoreSorterTest, ReplicationFactorTest, URLClassifyProcessorTest, DocumentBuilderTest, RequestLoggingTest, DateRangeFieldTest, TestFieldSortValues, SharedFSAutoReplicaFailoverTest, HdfsSyncSliceTest, TestRequestForwarding, TestRandomRequestDistribution, DocValuesTest, TestCharFilters, ClassificationUpdateProcessorFactoryTest, VersionInfoTest, TestCursorMarkWithoutUniqueKey, SpatialFilterTest, SimplePostToolTest, TestUniqueKeyFieldResource, TestJettySolrRunner, LoggingHandlerTest, MultiThreadedOCPTest, TestBM25SimilarityFactory, SpellCheckComponentTest, TestNonDefinedSimilarityFactory, TestQuerySenderListener, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [272/660 (1!)] on J1 in 91.86s, 20 tests, 1 failure <<< FAILURES!

[...truncated 63655 lines...]


Mime
View raw message