lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Høydahl <jan....@cominvent.com>
Subject Re: [JENKINS] Lucene-Solr-Tests-master - Build # 3155 - Failure
Date Thu, 24 Jan 2019 11:58:51 GMT
Reason for fail is probably timing issues on the real Jenkins runs, where asserting Metrics counts immediately after an action may miss the increment.
I decided to add a generic solution for this in SolrCloudAuthTestCase that will retry the assert after a 2s sleep before actually failing.
Note that this sleep is not applied in 90% of the cases, only when a race condition occurs. This should also make metrics asserting in e.g. BasicAuthIntegrationTest safer.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 24. jan. 2019 kl. 10:21 skrev Jan Høydahl <jan.asf@cominvent.com>:
> 
>> FAILED:  org.apache.solr.security.JWTAuthPluginIntegrationTest.testMetrics
> 
> Looking into this...
> 
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com <http://www.cominvent.com/>
> 
>> 24. jan. 2019 kl. 02:48 skrev Apache Jenkins Server <jenkins@builds.apache.org <mailto:jenkins@builds.apache.org>>:
>> 
>> Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/3155/ <https://builds.apache.org/job/Lucene-Solr-Tests-master/3155/>
>> 
>> 3 tests failed.
>> FAILED:  org.apache.solr.cloud.TestPrepRecovery.testLeaderUnloaded
>> 
>> Error Message:
>> Error from server at http://127.0.0.1:34371/solr: <http://127.0.0.1:34371/solr:> At least one of the node(s) specified [127.0.0.1:34371_solr] are not currently active in [], no action taken.
>> 
>> Stack Trace:
>> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34371/solr: <http://127.0.0.1:34371/solr:> At least one of the node(s) specified [127.0.0.1:34371_solr] are not currently active in [], no action taken.
>> 	at __randomizedtesting.SeedInfo.seed([49AF66F46CA26568:1A087502918E8B6A]:0)
>> 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:650)
>> 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:256)
>> 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:245)
>> 	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
>> 	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
>> 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:213)
>> 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1110)
>> 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
>> 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
>> 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)
>> 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:224)
>> 	at org.apache.solr.cloud.TestPrepRecovery.testLeaderUnloaded(TestPrepRecovery.java:73)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> 	at java.lang.reflect.Method.invoke(Method.java:498)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
>> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
>> 	at java.lang.Thread.run(Thread.java:748)
>> 
>> 
>> FAILED:  org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testSearchRate
>> 
>> Error Message:
>> The trigger did not start in time
>> 
>> Stack Trace:
>> java.lang.AssertionError: The trigger did not start in time
>> 	at __randomizedtesting.SeedInfo.seed([49AF66F46CA26568:14E7787DA364C327]:0)
>> 	at org.junit.Assert.fail(Assert.java:88)
>> 	at org.junit.Assert.assertTrue(Assert.java:41)
>> 	at org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testSearchRate(TestSimTriggerIntegration.java:1369)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> 	at java.lang.reflect.Method.invoke(Method.java:498)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
>> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
>> 	at java.lang.Thread.run(Thread.java:748)
>> 
>> 
>> FAILED:  org.apache.solr.security.JWTAuthPluginIntegrationTest.testMetrics
>> 
>> Error Message:
>> Expected metric minimums for prefix SECURITY./authentication.: {failMissingCredentials=1, authenticated=4, passThrough=4, failWrongCredentials=1, requests=10, errors=0}, but got: {failMissingCredentials=1, authenticated=4, passThrough=4, totalTime=89321132, failWrongCredentials=0, requestTimes=497, requests=10, errors=0}
>> 
>> Stack Trace:
>> java.lang.AssertionError: Expected metric minimums for prefix SECURITY./authentication.: {failMissingCredentials=1, authenticated=4, passThrough=4, failWrongCredentials=1, requests=10, errors=0}, but got: {failMissingCredentials=1, authenticated=4, passThrough=4, totalTime=89321132, failWrongCredentials=0, requestTimes=497, requests=10, errors=0}
>> 	at __randomizedtesting.SeedInfo.seed([49AF66F46CA26568:B7BDB45670A812DB]:0)
>> 	at org.junit.Assert.fail(Assert.java:88)
>> 	at org.junit.Assert.assertTrue(Assert.java:41)
>> 	at org.apache.solr.cloud.SolrCloudAuthTestCase.assertAuthMetricsMinimums(SolrCloudAuthTestCase.java:124)
>> 	at org.apache.solr.cloud.SolrCloudAuthTestCase.assertAuthMetricsMinimums(SolrCloudAuthTestCase.java:80)
>> 	at org.apache.solr.security.JWTAuthPluginIntegrationTest.testMetrics(JWTAuthPluginIntegrationTest.java:158)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> 	at java.lang.reflect.Method.invoke(Method.java:498)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
>> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
>> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
>> 	at java.lang.Thread.run(Thread.java:748)
>> 
>> 
>> 
>> 
>> Build Log:
>> [...truncated 13426 lines...]
>>   [junit4] Suite: org.apache.solr.cloud.TestPrepRecovery
>>   [junit4]   2> 1406342 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
>>   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/init-core-data-001
>>   [junit4]   2> 1406359 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
>>   [junit4]   2> 1406361 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
>>   [junit4]   2> 1406362 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001
>>   [junit4]   2> 1406396 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
>>   [junit4]   2> 1406440 INFO  (ZkTestServer Run Thread) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
>>   [junit4]   2> 1406440 INFO  (ZkTestServer Run Thread) [    ] o.a.s.c.ZkTestServer Starting server
>>   [junit4]   2> 1406537 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:39369
>>   [junit4]   2> 1406537 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:39369
>>   [junit4]   2> 1406537 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 39369
>>   [junit4]   2> 1406651 INFO  (zkConnectionManagerCallback-2690-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1406740 INFO  (zkConnectionManagerCallback-2692-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1406772 INFO  (zkConnectionManagerCallback-2694-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1406786 WARN  (jetty-launcher-2695-thread-1) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
>>   [junit4]   2> 1406786 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
>>   [junit4]   2> 1406786 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
>>   [junit4]   2> 1406786 INFO  (jetty-launcher-2695-thread-1) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
>>   [junit4]   2> 1406804 WARN  (jetty-launcher-2695-thread-2) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
>>   [junit4]   2> 1406805 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
>>   [junit4]   2> 1406805 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
>>   [junit4]   2> 1406805 INFO  (jetty-launcher-2695-thread-2) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
>>   [junit4]   2> 1406865 INFO  (jetty-launcher-2695-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
>>   [junit4]   2> 1406865 INFO  (jetty-launcher-2695-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
>>   [junit4]   2> 1406865 INFO  (jetty-launcher-2695-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
>>   [junit4]   2> 1406866 INFO  (jetty-launcher-2695-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3d2a9fc1{/solr,null,AVAILABLE}
>>   [junit4]   2> 1406951 INFO  (jetty-launcher-2695-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
>>   [junit4]   2> 1406951 INFO  (jetty-launcher-2695-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
>>   [junit4]   2> 1406951 INFO  (jetty-launcher-2695-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
>>   [junit4]   2> 1406952 INFO  (jetty-launcher-2695-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1bc0ea55{/solr,null,AVAILABLE}
>>   [junit4]   2> 1406988 INFO  (jetty-launcher-2695-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4f23a3ba{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:44188}
>>   [junit4]   2> 1406988 INFO  (jetty-launcher-2695-thread-2) [    ] o.e.j.s.Server Started @1407027ms
>>   [junit4]   2> 1406988 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44188}
>>   [junit4]   2> 1406989 ERROR (jetty-launcher-2695-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
>>   [junit4]   2> 1406989 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
>>   [junit4]   2> 1406989 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 9.0.0
>>   [junit4]   2> 1406989 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
>>   [junit4]   2> 1406989 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
>>   [junit4]   2> 1406989 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-01-24T00:04:02.909Z
>>   [junit4]   2> 1407015 INFO  (jetty-launcher-2695-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3b61f52a{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:37276}
>>   [junit4]   2> 1407015 INFO  (jetty-launcher-2695-thread-1) [    ] o.e.j.s.Server Started @1407054ms
>>   [junit4]   2> 1407015 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37276}
>>   [junit4]   2> 1407015 ERROR (jetty-launcher-2695-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
>>   [junit4]   2> 1407016 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
>>   [junit4]   2> 1407016 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 9.0.0
>>   [junit4]   2> 1407016 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
>>   [junit4]   2> 1407016 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
>>   [junit4]   2> 1407016 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-01-24T00:04:02.936Z
>>   [junit4]   2> 1407149 INFO  (zkConnectionManagerCallback-2699-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1407173 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>>   [junit4]   2> 1407177 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
>>   [junit4]   2> 1407177 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
>>   [junit4]   2> 1407177 INFO  (zkConnectionManagerCallback-2697-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1407191 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e, but no JMX reporters were configured - adding default JMX reporter.
>>   [junit4]   2> 1407206 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>>   [junit4]   2> 1407208 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
>>   [junit4]   2> 1407209 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
>>   [junit4]   2> 1407222 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e, but no JMX reporters were configured - adding default JMX reporter.
>>   [junit4]   2> 1408207 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
>>   [junit4]   2> 1408410 WARN  (jetty-launcher-2695-thread-1) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@6d5cb92d[provider=null,keyStore=null,trustStore=null]
>>   [junit4]   2> 1408556 WARN  (jetty-launcher-2695-thread-1) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@756de65[provider=null,keyStore=null,trustStore=null]
>>   [junit4]   2> 1408557 INFO  (jetty-launcher-2695-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39369/solr
>>   [junit4]   2> 1408572 INFO  (zkConnectionManagerCallback-2705-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1408644 INFO  (zkConnectionManagerCallback-2707-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1408902 INFO  (jetty-launcher-2695-thread-1) [n:127.0.0.1:37276_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37276_solr
>>   [junit4]   2> 1408903 INFO  (jetty-launcher-2695-thread-1) [n:127.0.0.1:37276_solr    ] o.a.s.c.Overseer Overseer (id=74064676197105670-127.0.0.1:37276_solr-n_0000000000) starting
>>   [junit4]   2> 1408923 INFO  (zkConnectionManagerCallback-2714-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1408938 INFO  (jetty-launcher-2695-thread-1) [n:127.0.0.1:37276_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39369/solr ready
>>   [junit4]   2> 1408939 INFO  (OverseerStateUpdate-74064676197105670-127.0.0.1:37276_solr-n_0000000000) [n:127.0.0.1:37276_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37276_solr
>>   [junit4]   2> 1408958 INFO  (jetty-launcher-2695-thread-1) [n:127.0.0.1:37276_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37276_solr
>>   [junit4]   2> 1408976 INFO  (zkCallback-2713-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
>>   [junit4]   2> 1408993 INFO  (zkCallback-2706-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
>>   [junit4]   2> 1409102 INFO  (jetty-launcher-2695-thread-1) [n:127.0.0.1:37276_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
>>   [junit4]   2> 1409322 INFO  (jetty-launcher-2695-thread-1) [n:127.0.0.1:37276_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e
>>   [junit4]   2> 1409468 INFO  (jetty-launcher-2695-thread-1) [n:127.0.0.1:37276_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e
>>   [junit4]   2> 1409468 INFO  (jetty-launcher-2695-thread-1) [n:127.0.0.1:37276_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e
>>   [junit4]   2> 1409469 INFO  (jetty-launcher-2695-thread-1) [n:127.0.0.1:37276_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/.
>>   [junit4]   2> 1410798 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
>>   [junit4]   2> 1410834 WARN  (jetty-launcher-2695-thread-2) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@1e01a118[provider=null,keyStore=null,trustStore=null]
>>   [junit4]   2> 1410943 WARN  (jetty-launcher-2695-thread-2) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@486a45b1[provider=null,keyStore=null,trustStore=null]
>>   [junit4]   2> 1410944 INFO  (jetty-launcher-2695-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39369/solr
>>   [junit4]   2> 1411039 INFO  (zkConnectionManagerCallback-2721-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1411104 INFO  (zkConnectionManagerCallback-2723-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1411145 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
>>   [junit4]   2> 1411199 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:44188_solr as DOWN
>>   [junit4]   2> 1411200 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
>>   [junit4]   2> 1411200 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44188_solr
>>   [junit4]   2> 1411270 INFO  (zkCallback-2713-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
>>   [junit4]   2> 1411270 INFO  (zkCallback-2706-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
>>   [junit4]   2> 1411272 INFO  (zkCallback-2722-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
>>   [junit4]   2> 1411582 INFO  (zkConnectionManagerCallback-2730-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1411583 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
>>   [junit4]   2> 1411584 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39369/solr ready
>>   [junit4]   2> 1411598 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
>>   [junit4]   2> 1411766 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e
>>   [junit4]   2> 1411820 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e
>>   [junit4]   2> 1411820 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e
>>   [junit4]   2> 1411874 INFO  (jetty-launcher-2695-thread-2) [n:127.0.0.1:44188_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/.
>>   [junit4]   2> 1412238 INFO  (zkConnectionManagerCallback-2736-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1412239 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
>>   [junit4]   2> 1412240 INFO  (SUITE-TestPrepRecovery-seed#[49AF66F46CA26568]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39369/solr ready
>>   [junit4]   2> 1412567 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderUnloaded
>>   [junit4]   2> 1412583 INFO  (qtp738136349-7783) [n:127.0.0.1:37276_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&name=testLeaderUnloaded&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
>>   [junit4]   2> 1412619 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testLeaderUnloaded
>>   [junit4]   2> 1412620 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/testLeaderUnloaded.AUTOCREATED/protwords.txt
>>   [junit4]   2> 1412621 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/testLeaderUnloaded.AUTOCREATED/managed-schema
>>   [junit4]   2> 1412639 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/testLeaderUnloaded.AUTOCREATED/solrconfig.xml
>>   [junit4]   2> 1412641 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/testLeaderUnloaded.AUTOCREATED/synonyms.txt
>>   [junit4]   2> 1412641 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/testLeaderUnloaded.AUTOCREATED/stopwords.txt
>>   [junit4]   2> 1412642 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/testLeaderUnloaded.AUTOCREATED/params.json
>>   [junit4]   2> 1412676 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/contractions_it.txt
>>   [junit4]   2> 1412677 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_no.txt
>>   [junit4]   2> 1412692 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_hy.txt
>>   [junit4]   2> 1412693 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stemdict_nl.txt
>>   [junit4]   2> 1412694 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/contractions_ca.txt
>>   [junit4]   2> 1412695 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_id.txt
>>   [junit4]   2> 1412751 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_ga.txt
>>   [junit4]   2> 1412752 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_da.txt
>>   [junit4]   2> 1412786 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_hi.txt
>>   [junit4]   2> 1412787 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_ja.txt
>>   [junit4]   2> 1412788 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_pt.txt
>>   [junit4]   2> 1412802 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_hu.txt
>>   [junit4]   2> 1412803 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_el.txt
>>   [junit4]   2> 1412805 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_ru.txt
>>   [junit4]   2> 1412822 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_tr.txt
>>   [junit4]   2> 1412823 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_ar.txt
>>   [junit4]   2> 1412824 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/userdict_ja.txt
>>   [junit4]   2> 1412858 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_it.txt
>>   [junit4]   2> 1412859 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_eu.txt
>>   [junit4]   2> 1412860 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_cz.txt
>>   [junit4]   2> 1412861 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stoptags_ja.txt
>>   [junit4]   2> 1412895 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/contractions_fr.txt
>>   [junit4]   2> 1412896 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_fa.txt
>>   [junit4]   2> 1412897 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_de.txt
>>   [junit4]   2> 1412898 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/contractions_ga.txt
>>   [junit4]   2> 1412911 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_ca.txt
>>   [junit4]   2> 1412913 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_nl.txt
>>   [junit4]   2> 1412914 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_bg.txt
>>   [junit4]   2> 1412932 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_es.txt
>>   [junit4]   2> 1412933 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_en.txt
>>   [junit4]   2> 1412934 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_lv.txt
>>   [junit4]   2> 1412935 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_th.txt
>>   [junit4]   2> 1412985 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_ro.txt
>>   [junit4]   2> 1412986 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_fr.txt
>>   [junit4]   2> 1412987 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_sv.txt
>>   [junit4]   2> 1413020 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_fi.txt
>>   [junit4]   2> 1413022 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/hyphenations_ga.txt
>>   [junit4]   2> 1413022 INFO  (OverseerThreadFactory-2967-thread-1-processing-n:127.0.0.1:37276_solr) [n:127.0.0.1:37276_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/testLeaderUnloaded.AUTOCREATED/lang/stopwords_gl.txt
>>   [junit4]   2> 1413321 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=166
>>   [junit4]   2> 1413322 INFO  (qtp738136349-7786) [n:127.0.0.1:37276_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>>   [junit4]   2> 1413323 INFO  (qtp738136349-7782) [n:127.0.0.1:37276_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>>   [junit4]   2> 1413324 INFO  (qtp738136349-7784) [n:127.0.0.1:37276_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>>   [junit4]   2> 1413373 INFO  (qtp738136349-7784) [n:127.0.0.1:37276_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=36
>>   [junit4]   2> 1413407 INFO  (qtp812348173-7793) [n:127.0.0.1:44188_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>>   [junit4]   2> 1413408 INFO  (qtp812348173-7791) [n:127.0.0.1:44188_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>>   [junit4]   2> 1413412 INFO  (qtp812348173-7794) [n:127.0.0.1:44188_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>>   [junit4]   2> 1413446 INFO  (qtp812348173-7795) [n:127.0.0.1:44188_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=33
>>   [junit4]   2> 1413447 INFO  (qtp812348173-7795) [n:127.0.0.1:44188_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>>   [junit4]   2> 1413624 INFO  (OverseerStateUpdate-74064676197105670-127.0.0.1:37276_solr-n_0000000000) [n:127.0.0.1:37276_solr    ] o.a.s.c.o.SliceMutator createReplica() {
>>   [junit4]   2>   "operation":"ADDREPLICA",
>>   [junit4]   2>   "collection":"testLeaderUnloaded",
>>   [junit4]   2>   "shard":"shard1",
>>   [junit4]   2>   "core":"testLeaderUnloaded_shard1_replica_n1",
>>   [junit4]   2>   "state":"down",
>>   [junit4]   2>   "base_url":"http://127.0.0.1:44188/solr <http://127.0.0.1:44188/solr>",
>>   [junit4]   2>   "type":"NRT",
>>   [junit4]   2>   "waitForFinalState":"false"} 
>>   [junit4]   2> 1413696 INFO  (OverseerStateUpdate-74064676197105670-127.0.0.1:37276_solr-n_0000000000) [n:127.0.0.1:37276_solr    ] o.a.s.c.o.SliceMutator createReplica() {
>>   [junit4]   2>   "operation":"ADDREPLICA",
>>   [junit4]   2>   "collection":"testLeaderUnloaded",
>>   [junit4]   2>   "shard":"shard1",
>>   [junit4]   2>   "core":"testLeaderUnloaded_shard1_replica_n2",
>>   [junit4]   2>   "state":"down",
>>   [junit4]   2>   "base_url":"http://127.0.0.1:37276/solr <http://127.0.0.1:37276/solr>",
>>   [junit4]   2>   "type":"NRT",
>>   [junit4]   2>   "waitForFinalState":"false"} 
>>   [junit4]   2> 1414030 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr    x:testLeaderUnloaded_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=testLeaderUnloaded.AUTOCREATED&newCollection=true&name=testLeaderUnloaded_shard1_replica_n1&action=CREATE&numShards=1&collection=testLeaderUnloaded&shard=shard1&wt=javabin&version=2&replicaType=NRT
>>   [junit4]   2> 1414066 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr    x:testLeaderUnloaded_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=testLeaderUnloaded.AUTOCREATED&newCollection=true&name=testLeaderUnloaded_shard1_replica_n2&action=CREATE&numShards=1&collection=testLeaderUnloaded&shard=shard1&wt=javabin&version=2&replicaType=NRT
>>   [junit4]   2> 1414066 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr    x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
>>   [junit4]   2> 1415489 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0 
>>   [junit4]   2> 1415489 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
>>   [junit4]   2> 1415490 WARN  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../contrib/extraction/lib
>>   [junit4]   2> 1415490 WARN  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist
>>   [junit4]   2> 1415490 WARN  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../contrib/clustering/lib
>>   [junit4]   2> 1415490 WARN  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist
>>   [junit4]   2> 1415490 WARN  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../contrib/langid/lib
>>   [junit4]   2> 1415490 WARN  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist
>>   [junit4]   2> 1415490 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0 
>>   [junit4]   2> 1415491 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
>>   [junit4]   2> 1415491 WARN  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../contrib/velocity/lib
>>   [junit4]   2> 1415491 WARN  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist
>>   [junit4]   2> 1415491 WARN  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist filtered by solr-ltr-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1/../../../../dist
>>   [junit4]   2> 1415491 WARN  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../contrib/extraction/lib
>>   [junit4]   2> 1415541 WARN  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist
>>   [junit4]   2> 1415541 WARN  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../contrib/clustering/lib
>>   [junit4]   2> 1415541 WARN  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist
>>   [junit4]   2> 1415541 WARN  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../contrib/langid/lib
>>   [junit4]   2> 1415542 WARN  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist
>>   [junit4]   2> 1415542 WARN  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../contrib/velocity/lib
>>   [junit4]   2> 1415542 WARN  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist
>>   [junit4]   2> 1415542 WARN  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist filtered by solr-ltr-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2/../../../../dist
>>   [junit4]   2> 1415564 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
>>   [junit4]   2> 1415579 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
>>   [junit4]   2> 1415867 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.s.IndexSchema [testLeaderUnloaded_shard1_replica_n2] Schema name=default-config
>>   [junit4]   2> 1415944 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.s.IndexSchema [testLeaderUnloaded_shard1_replica_n1] Schema name=default-config
>>   [junit4]   2> 1423176 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
>>   [junit4]   2> 1423250 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
>>   [junit4]   2> 1423733 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testLeaderUnloaded_shard1_replica_n1' using configuration from collection testLeaderUnloaded, trusted=true
>>   [junit4]   2> 1423734 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.testLeaderUnloaded.shard1.replica_n1' (registry 'solr.core.testLeaderUnloaded.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e
>>   [junit4]   2> 1423734 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrCore [[testLeaderUnloaded_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/testLeaderUnloaded_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node2/./testLeaderUnloaded_shard1_replica_n1/data/]
>>   [junit4]   2> 1424004 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
>>   [junit4]   2> 1424560 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
>>   [junit4]   2> 1424560 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
>>   [junit4]   2> 1424562 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; 
>>   [junit4]   2> 1424562 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
>>   [junit4]   2> 1424653 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3fdaf2c5[testLeaderUnloaded_shard1_replica_n1] main]
>>   [junit4]   2> 1424654 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/testLeaderUnloaded.AUTOCREATED
>>   [junit4]   2> 1424655 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/testLeaderUnloaded.AUTOCREATED
>>   [junit4]   2> 1424655 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/testLeaderUnloaded.AUTOCREATED/managed-schema
>>   [junit4]   2> 1424655 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
>>   [junit4]   2> 1424656 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
>>   [junit4]   2> 1424656 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=default,field=_text_,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
>>   [junit4]   2> 1424670 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
>>   [junit4]   2> 1424670 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1623497911136419840
>>   [junit4]   2> 1424672 INFO  (searcherExecutor-2979-thread-1-processing-n:127.0.0.1:44188_solr x:testLeaderUnloaded_shard1_replica_n1 c:testLeaderUnloaded s:shard1 r:core_node3) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@3fdaf2c5[testLeaderUnloaded_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
>>   [junit4]   2> 1424672 INFO  (searcherExecutor-2979-thread-1-processing-n:127.0.0.1:44188_solr x:testLeaderUnloaded_shard1_replica_n1 c:testLeaderUnloaded s:shard1 r:core_node3) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener done.
>>   [junit4]   2> 1424672 INFO  (searcherExecutor-2979-thread-1-processing-n:127.0.0.1:44188_solr x:testLeaderUnloaded_shard1_replica_n1 c:testLeaderUnloaded s:shard1 r:core_node3) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
>>   [junit4]   2> 1424693 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testLeaderUnloaded/terms/shard1 to Terms{values={core_node3=0}, version=0}
>>   [junit4]   2> 1424693 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testLeaderUnloaded/leaders/shard1
>>   [junit4]   2> 1424727 INFO  (searcherExecutor-2979-thread-1-processing-n:127.0.0.1:44188_solr x:testLeaderUnloaded_shard1_replica_n1 c:testLeaderUnloaded s:shard1 r:core_node3) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SolrCore [testLeaderUnloaded_shard1_replica_n1] Registered new searcher Searcher@3fdaf2c5[testLeaderUnloaded_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
>>   [junit4]   2> 1424743 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=14999ms
>>   [junit4]   2> 1424823 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testLeaderUnloaded_shard1_replica_n2' using configuration from collection testLeaderUnloaded, trusted=true
>>   [junit4]   2> 1424824 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.testLeaderUnloaded.shard1.replica_n2' (registry 'solr.core.testLeaderUnloaded.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7eb40f1e
>>   [junit4]   2> 1424824 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrCore [[testLeaderUnloaded_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/testLeaderUnloaded_shard1_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestPrepRecovery_49AF66F46CA26568-001/tempDir-001/node1/./testLeaderUnloaded_shard1_replica_n2/data/]
>>   [junit4]   2> 1425066 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
>>   [junit4]   2> 1425620 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
>>   [junit4]   2> 1425620 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
>>   [junit4]   2> 1425635 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; 
>>   [junit4]   2> 1425635 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
>>   [junit4]   2> 1425671 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7dabbd13[testLeaderUnloaded_shard1_replica_n2] main]
>>   [junit4]   2> 1425678 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/testLeaderUnloaded.AUTOCREATED
>>   [junit4]   2> 1425678 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/testLeaderUnloaded.AUTOCREATED
>>   [junit4]   2> 1425679 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/testLeaderUnloaded.AUTOCREATED/managed-schema
>>   [junit4]   2> 1425679 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
>>   [junit4]   2> 1425679 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
>>   [junit4]   2> 1425680 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.s.DirectSolrSpellChecker init: {name=default,field=_text_,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
>>   [junit4]   2> 1425681 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
>>   [junit4]   2> 1425681 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1623497912196530176
>>   [junit4]   2> 1425726 INFO  (searcherExecutor-2984-thread-1-processing-n:127.0.0.1:37276_solr x:testLeaderUnloaded_shard1_replica_n2 c:testLeaderUnloaded s:shard1 r:core_node4) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@7dabbd13[testLeaderUnloaded_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
>>   [junit4]   2> 1425726 INFO  (searcherExecutor-2984-thread-1-processing-n:127.0.0.1:37276_solr x:testLeaderUnloaded_shard1_replica_n2 c:testLeaderUnloaded s:shard1 r:core_node4) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.QuerySenderListener QuerySenderListener done.
>>   [junit4]   2> 1425727 INFO  (searcherExecutor-2984-thread-1-processing-n:127.0.0.1:37276_solr x:testLeaderUnloaded_shard1_replica_n2 c:testLeaderUnloaded s:shard1 r:core_node4) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
>>   [junit4]   2> 1425727 INFO  (searcherExecutor-2984-thread-1-processing-n:127.0.0.1:37276_solr x:testLeaderUnloaded_shard1_replica_n2 c:testLeaderUnloaded s:shard1 r:core_node4) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.SolrCore [testLeaderUnloaded_shard1_replica_n2] Registered new searcher Searcher@7dabbd13[testLeaderUnloaded_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
>>   [junit4]   2> 1425731 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testLeaderUnloaded/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
>>   [junit4]   2> 1425731 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testLeaderUnloaded/leaders/shard1
>>   [junit4]   2> 1425901 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
>>   [junit4]   2> 1425901 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
>>   [junit4]   2> 1425901 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44188/solr/testLeaderUnloaded_shard1_replica_n1/ <http://127.0.0.1:44188/solr/testLeaderUnloaded_shard1_replica_n1/>
>>   [junit4]   2> 1425902 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=testLeaderUnloaded_shard1_replica_n1 url=http://127.0.0.1:44188/solr <http://127.0.0.1:44188/solr> START replicas=[http://127.0.0.1:37276/solr/testLeaderUnloaded_shard1_replica_n2/ <http://127.0.0.1:37276/solr/testLeaderUnloaded_shard1_replica_n2/>] nUpdates=100
>>   [junit4]   2> 1425917 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=testLeaderUnloaded_shard1_replica_n1 url=http://127.0.0.1:44188/solr <http://127.0.0.1:44188/solr> DONE.  We have no versions.  sync failed.
>>   [junit4]   2> 1425919 INFO  (qtp738136349-7782) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.c.S.Request [testLeaderUnloaded_shard1_replica_n2]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
>>   [junit4]   2> 1425920 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
>>   [junit4]   2> 1425920 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
>>   [junit4]   2> 1425920 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testLeaderUnloaded/leaders/shard1/leader after winning as /collections/testLeaderUnloaded/leader_elect/shard1/election/74064676197105673-core_node3-n_0000000000
>>   [junit4]   2> 1425990 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44188/solr/testLeaderUnloaded_shard1_replica_n1/ <http://127.0.0.1:44188/solr/testLeaderUnloaded_shard1_replica_n1/> shard1
>>   [junit4]   2> 1426124 INFO  (zkCallback-2722-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testLeaderUnloaded/state.json] for collection [testLeaderUnloaded] has occurred - updating... (live nodes size: [2])
>>   [junit4]   2> 1426152 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
>>   [junit4]   2> 1426154 INFO  (qtp812348173-7792) [n:127.0.0.1:44188_solr c:testLeaderUnloaded s:shard1 r:core_node3 x:testLeaderUnloaded_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=testLeaderUnloaded.AUTOCREATED&newCollection=true&name=testLeaderUnloaded_shard1_replica_n1&action=CREATE&numShards=1&collection=testLeaderUnloaded&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=12124
>>   [junit4]   2> 1426270 INFO  (zkCallback-2722-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testLeaderUnloaded/state.json] for collection [testLeaderUnloaded] has occurred - updating... (live nodes size: [2])
>>   [junit4]   2> 1426290 INFO  (zkCallback-2722-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testLeaderUnloaded/state.json] for collection [testLeaderUnloaded] has occurred - updating... (live nodes size: [2])
>>   [junit4]   2> 1426797 INFO  (qtp738136349-7785) [n:127.0.0.1:37276_solr c:testLeaderUnloaded s:shard1 r:core_node4 x:testLeaderUnloaded_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=testLeaderUnloaded.AUTOCREATED&newCollection=true&name=testLeaderUnloaded_shard1_replica_n2&action=CREATE&numShards=1&collection=testLeaderUnloaded&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=12731
>>   [junit4]   2> 1426813 INFO  (qtp738136349-7783) [n:127.0.0.1:37276_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
>>   [junit4]   2> 1426850 INFO  (zkCallback-2722-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testLeaderUnloaded/state.json] for collection [testLeaderUnloaded] has occurred - updating... (live nodes size: [2])
>>   [junit4]   2> 1426850 INFO  (zkCallback-2722-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testLeaderUnloaded/state.json] for collection [testLeaderUnloaded] has occurred - updating... (live nodes size: [2])
>>   [junit4]   2> 1426850 INFO  (zkCallback-2706-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testLeaderUnloaded/state.json] for collection [testLeaderUnloaded] has occurred - updating... (live nodes size: [2])
>>   [junit4]   2> 1426867 INFO  (zkCallback-2706-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testLeaderUnloaded/state.json] for collection [testLeaderUnloaded] has occurred - updating... (live nodes size: [2])
>>   [junit4]   2> 1426868 INFO  (qtp738136349-7783) [n:127.0.0.1:37276_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&name=testLeaderUnloaded&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=14284
>>   [junit4]   2> 1426888 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.c.SolrCloudTestCase active replica count: 2 expected replica count: 2
>>   [junit4]   2> 1426888 WARN  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
>>   [junit4]   2> 1426889 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
>>   [junit4]   2> 1426889 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
>>   [junit4]   2> 1426889 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
>>   [junit4]   2> 1426991 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
>>   [junit4]   2> 1426991 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
>>   [junit4]   2> 1426991 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
>>   [junit4]   2> 1426991 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1d8b0e2e{/solr,null,AVAILABLE}
>>   [junit4]   2> 1426992 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@a53d776{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:34371}
>>   [junit4]   2> 1426992 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.e.j.s.Server Started @1427031ms
>>   [junit4]   2> 1426992 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34371}
>>   [junit4]   2> 1426992 ERROR (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
>>   [junit4]   2> 1426992 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
>>   [junit4]   2> 1426993 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 9.0.0
>>   [junit4]   2> 1426993 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
>>   [junit4]   2> 1426993 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
>>   [junit4]   2> 1426993 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-01-24T00:04:22.913Z
>>   [junit4]   2> 1427027 INFO  (zkConnectionManagerCallback-2740-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
>>   [junit4]   2> 1427028 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>>   [junit4]   2> 1427044 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
>>   [junit4]   2> 1427044 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
>>   [junit4]   2> 1427045 INFO  (TEST-TestPrepRecovery.testLeaderUnloaded-seed#[49AF66F46CA26568]) [    ] o.a.s.c.SolrXmlConfig MBean server found
>> 
>> [...truncated too long message...]
>> 
>> orizationPlugin, HdfsDirectoryFactoryTest, ImplicitSnitchTest, TestCloudManagedSchema, CollectionsAPIAsyncDistributedZkTest, TestPerFieldSimilarityWithDefaultOverride, SchemaVersionSpecificBehaviorTest, SortSpecParsingTest, TestSolrCLIRunExample, SignatureUpdateProcessorFactoryTest, RangeFacetCloudTest, TestUnifiedSolrHighlighter, MultiThreadedOCPTest, ShardsWhitelistTest, TestNumericTerms32, TestDownShardTolerantSearch, TestStressReorder, TermVectorComponentDistributedTest, TestSSLRandomization, CoreAdminRequestStatusTest, MoveReplicaHDFSTest, BinaryUpdateRequestHandlerTest, Tagger2Test, ShardRoutingCustomTest, ActionThrottleTest, TestSimTriggerIntegration, RollingRestartTest, OverseerTest, IndexBasedSpellCheckerTest, TestRawResponseWriter, BadIndexSchemaTest, HdfsChaosMonkeyNothingIsSafeTest, JavabinLoaderTest, TestDefaultStatsCache, TestCustomStream, TestFieldCacheVsDocValues, DistributedExpandComponentTest, TestRestoreCore, TestFieldSortValues, SolrJmxReporterTest, TestSolrQueryResponse, DistribJoinFromCollectionTest, TestMinHashQParser, TestClusterStateMutator, MigrateRouteKeyTest, TestPullReplica, TestConfigSetImmutable, TestCharFilters, TestWithCollection, TestImpersonationWithHadoopAuth, SliceStateTest, LeaderElectionContextKeyTest, TestMergePolicyConfig, TestJsonFacetsStatsParsing, FullHLLTest, PolyFieldTest, CurrencyRangeFacetCloudTest, TestSolrConfigHandlerCloud, BufferStoreTest, TestReplicationHandler, TestSubQueryTransformer, FieldAnalysisRequestHandlerTest, TestStreamBody, TestCloudInspectUtil, DeleteStatusTest, SuggesterTest, TestLuceneIndexBackCompat, TestHdfsCloudBackupRestore, TestPayloadScoreQParserPlugin, TestCollectionAPIs, TriggerCooldownIntegrationTest, TestSortableTextField, WordBreakSolrSpellCheckerTest, PrimitiveFieldTypeTest, SimpleFacetsTest, DistributedSuggestComponentTest, HdfsAutoAddReplicasIntegrationTest, NodeAddedTriggerTest, HdfsRecoverLeaseTest, TestSimNodeAddedTrigger, TestLegacyTerms, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestDistributedMissingSort, TestDocumentBuilder, TestHighlightDedupGrouping, TestJoin, TestRandomFaceting, TestSimpleTrackingShardHandler, TestDeprecatedFilters, TestLuceneMatchVersion, TestReversedWildcardFilterFactory, TestWordDelimiterFilterFactory, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, AddReplicaTest, AsyncCallRequestStatusResponseTest, BasicDistributedZk2Test, CollectionsAPISolrJTest, LeaderFailoverAfterPartitionTest, MissingSegmentRecoveryTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, OverseerStatusTest, ReplaceNodeNoTargetTest, ReplaceNodeTest, SolrCloudExampleTest, TestCloudPseudoReturnFields, TestCloudSearcherWarming, TestPullReplicaErrorHandling, TestSolrCloudWithDelegationTokens, TestStressLiveNodes, ZkFailoverTest, ConcurrentDeleteAndCreateCollectionTest, CustomCollectionTest, SimpleCollectionCreateDeleteTest, TestLocalFSCloudBackupRestore, TestReplicaProperties, TestRequestStatusCollectionAPI, ComputePlanActionTest, MetricTriggerTest, NodeLostTriggerTest, ScheduledTriggerIntegrationTest, SearchRateTriggerIntegrationTest, SearchRateTriggerTest, HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest, HdfsRestartWhileUpdatingTest, HdfsSyncSliceTest, HdfsWriteToMultipleCollectionsTest, RulesTest, OpenCloseCoreStressTest, TestBadConfig, TestDirectoryFactory, TestDynamicLoading, TestSystemCollAutoCreate, V2ApiIntegrationTest, CoreAdminCreateDiscoverTest, CoreMergeIndexesAdminHandlerTest, PropertiesRequestHandlerTest, SecurityConfHandlerTest, TestConfigsApi, TestCoreAdminApis, ZookeeperStatusHandlerTest, BadComponentTest, CustomHighlightComponentTest, CustomTermsComponentTest, DistributedFacetPivotLargeTest, DistributedMLTComponentTest, QueryElevationComponentTest, TaggingAttributeTest, HighlighterTest, TestNumericTokenStream, SolrCoreMetricManagerTest, SolrShardReporterTest, TestWriterPerf, TestRawTransformer, TestSubQueryTransformerDistrib, TestCopyFieldCollectionResource, TestFieldTypeResource, TestSchemaNameResource, TestSchemaResource, TestSchemaVersionResource, TestSerializedLuceneMatchVersion, TestUniqueKeyFieldResource, TestManagedStopFilterFactory, CurrencyFieldTypeTest, DocValuesMultiTest, DocValuesTest, SynonymTokenizerTest, TestBinaryField, TestCloudSchemaless, TestFilteredDocIdSet, TestFiltering, TestLRUCache, TestMissingGroups, TestQueryUtils, TestRangeQuery, TestStandardQParsers, TestStressVersions, TestXmlQParserPlugin, SpatialHeatmapFacetsTest, BlockJoinFacetRandomTest, BlockJoinFacetSimpleTest, GraphQueryTest, TestCloudNestedDocsSort, SimpleMLTQParserTest, TestBM25SimilarityFactory, TestClassicSimilarityFactory, TestDFRSimilarityFactory, TestIBSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestLegacyBM25SimilarityFactory, TestNonDefinedSimilarityFactory, TestSweetSpotSimilarityFactory, TestDistribIDF, TestExactSharedStatsCache, JWTAuthPluginIntegrationTest]
>>   [junit4] Completed [761/841 (3!)] on J2 in 31.64s, 3 tests, 1 failure <<< FAILURES!
>> 
>> [...truncated 46104 lines...]
>> -ecj-javadoc-lint-tests:
>>    [mkdir] Created dir: /tmp/ecj113807969
>> [ecj-lint] Compiling 942 source files to /tmp/ecj113807969
>> [ecj-lint] invalid Class-Path header in manifest of jar file: /home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
>> [ecj-lint] invalid Class-Path header in manifest of jar file: /home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
>> [ecj-lint] ----------
>> [ecj-lint] 1. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test/org/apache/solr/analysis/TokenizerChainTest.java (at line 37)
>> [ecj-lint] 	TokenizerChain tokenizerChain = new TokenizerChain(
>> [ecj-lint] 	               ^^^^^^^^^^^^^^
>> [ecj-lint] Resource leak: 'tokenizerChain' is never closed
>> [ecj-lint] ----------
>> [ecj-lint] ----------
>> [ecj-lint] 2. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test/org/apache/solr/cloud/ZkNodePropsTest.java (at line 48)
>> [ecj-lint] 	new JavaBinCodec().marshal(zkProps.getProperties(), baos);
>> [ecj-lint] 	^^^^^^^^^^^^^^^^^^
>> [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
>> [ecj-lint] ----------
>> [ecj-lint] ----------
>> [ecj-lint] 3. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test/org/apache/solr/cloud/autoscaling/NodeMarkersRegistrationTest.java (at line 31)
>> [ecj-lint] 	import org.apache.lucene.util.LuceneTestCase.AwaitsFix;
>> [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> [ecj-lint] The import org.apache.lucene.util.LuceneTestCase.AwaitsFix is never used
>> [ecj-lint] ----------
>> [ecj-lint] ----------
>> [ecj-lint] 4. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java (at line 716)
>> [ecj-lint] 	throw new UnsupportedOperationException("must add at least 1 node first");
>> [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> [ecj-lint] Resource leak: 'queryRequest' is not closed at this location
>> [ecj-lint] ----------
>> [ecj-lint] 5. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java (at line 722)
>> [ecj-lint] 	throw new UnsupportedOperationException("must add at least 1 node first");
>> [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> [ecj-lint] Resource leak: 'queryRequest' is not closed at this location
>> [ecj-lint] ----------
>> [ecj-lint] ----------
>> [ecj-lint] 6. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test/org/apache/solr/response/TestBinaryResponseWriter.java (at line 75)
>> [ecj-lint] 	new JavaBinCodec(new BinaryResponseWriter.Resolver(null, null)).marshal(nl, baos);
>> [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
>> [ecj-lint] ----------
>> [ecj-lint] 7. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test/org/apache/solr/response/TestBinaryResponseWriter.java (at line 77)
>> [ecj-lint] 	nl = (NamedList) new JavaBinCodec().unmarshal(new ByteArrayInputStream(byteBuffer.array(), 0, byteBuffer.limit()));
>> [ecj-lint] 	                 ^^^^^^^^^^^^^^^^^^
>> [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
>> [ecj-lint] ----------
>> [ecj-lint] ----------
>> [ecj-lint] 8. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test/org/apache/solr/update/processor/DistributedUpdateProcessorTest.java (at line 39)
>> [ecj-lint] 	DistributedUpdateProcessor processor = new DistributedUpdateProcessor(
>> [ecj-lint] 	                           ^^^^^^^^^
>> [ecj-lint] Resource leak: 'processor' is never closed
>> [ecj-lint] ----------
>> [ecj-lint] 8 problems (1 error, 7 warnings)
>> 
>> BUILD FAILED
>> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:633: The following error occurred while executing this line:
>> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:101: The following error occurred while executing this line:
>> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:680: The following error occurred while executing this line:
>> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2099: The following error occurred while executing this line:
>> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2132: Compile failed; see the compiler error output for details.
>> 
>> Total time: 218 minutes 44 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
>> For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>


Mime
View raw message