lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 27 - Still Failing
Date Fri, 01 Apr 2016 08:27:08 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/27/

5 tests failed.
FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=18654, name=Thread-8978, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=18654, name=Thread-8978, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57895/collection1
	at __randomizedtesting.SeedInfo.seed([509C900EC7537C02]:0)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:644)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57895/collection1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:588)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:642)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:482)
	... 5 more


FAILED:  org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([509C900EC7537C02:D8C8AFD469AF11FA]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertTrue(Assert.java:54)
	at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:226)
	at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
replicaCount expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: replicaCount expected:<0> but was:<1>
	at __randomizedtesting.SeedInfo.seed([509C900EC7537C02:D8C8AFD469AF11FA]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testNoConfigSetExist(CollectionsAPIDistributedZkTest.java:601)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:166)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 9 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 9 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([509C900EC7537C02]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:248)
	at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=5942, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5942, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50456: collection already exists: awholynewstresscollection_collection0_1
	at __randomizedtesting.SeedInfo.seed([509C900EC7537C02]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1165)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:935)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1614)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:970)




Build Log:
[...truncated 11183 lines...]
   [junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/init-core-data-001
   [junit4]   2> 1    INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /is_s/
   [junit4]   2> 2795 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 5940 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 6905 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 6913 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 7104 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 7395 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_55331_hdfs____.7e1btx/webapp
   [junit4]   2> 7860 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 9044 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:55331
   [junit4]   2> 12400 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 12808 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 12829 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_33566_datanode____at4hj0/webapp
   [junit4]   2> 13019 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 13739 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33566
   [junit4]   2> 15430 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 15433 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 15464 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47450_datanode____byk7kj/webapp
   [junit4]   2> 15815 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 17004 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[509C900EC7537C02]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47450
   [junit4]   2> 18348 INFO  (IPC Server handler 5 on 39532) [    ] BlockStateChange BLOCK* processReport: from storage DS-f8b4d8ce-2678-4169-9a1b-0086cce9b41f node DatanodeRegistration(127.0.0.1, datanodeUuid=09f82c23-cf66-4a37-a4d6-b4c7b4a81c03, infoPort=33566, ipcPort=33397, storageInfo=lv=-56;cid=testClusterID;nsid=947570425;c=0), blocks: 0, hasStaleStorages: true, processing time: 2 msecs
   [junit4]   2> 18348 INFO  (IPC Server handler 5 on 39532) [    ] BlockStateChange BLOCK* processReport: from storage DS-a4e05ab5-37d1-4f6b-b16e-fb982642bcbb node DatanodeRegistration(127.0.0.1, datanodeUuid=09f82c23-cf66-4a37-a4d6-b4c7b4a81c03, infoPort=33566, ipcPort=33397, storageInfo=lv=-56;cid=testClusterID;nsid=947570425;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 18351 INFO  (IPC Server handler 6 on 39532) [    ] BlockStateChange BLOCK* processReport: from storage DS-c328ed96-157b-4603-bb76-0b06a20d777c node DatanodeRegistration(127.0.0.1, datanodeUuid=5c5e4189-c87c-4a5b-bde1-87b0439b1e34, infoPort=47450, ipcPort=53454, storageInfo=lv=-56;cid=testClusterID;nsid=947570425;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 18355 INFO  (IPC Server handler 6 on 39532) [    ] BlockStateChange BLOCK* processReport: from storage DS-090bac70-9130-4c20-8787-e8b3a2112c31 node DatanodeRegistration(127.0.0.1, datanodeUuid=5c5e4189-c87c-4a5b-bde1-87b0439b1e34, infoPort=47450, ipcPort=53454, storageInfo=lv=-56;cid=testClusterID;nsid=947570425;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 20226 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 20227 INFO  (Thread-111) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 20228 INFO  (Thread-111) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 20331 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.ZkTestServer start zk server on port:36355
   [junit4]   2> 20380 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 20456 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 20502 INFO  (zkCallback-1-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4af3adc0 name:ZooKeeperConnection Watcher:127.0.0.1:36355 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 20502 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 20506 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 20507 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 20533 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 20541 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 20550 INFO  (zkCallback-2-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@621cdace name:ZooKeeperConnection Watcher:127.0.0.1:36355/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 20550 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 20551 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 20555 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 20564 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 20569 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 20573 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 20576 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 20587 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 20597 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 20597 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 20602 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 20602 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 20608 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 20608 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 20614 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 20614 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 20618 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 20618 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 20621 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 20621 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 20625 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 20625 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 20627 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 20627 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 20795 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 20796 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 20806 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 20807 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 20969 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/cores/collection1
   [junit4]   2> 20989 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.e.j.u.log Logging initialized @27012ms
   [junit4]   2> 21133 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 21213 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f422113{/is_s,null,AVAILABLE}
   [junit4]   2> 21229 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.e.j.s.ServerConnector Started ServerConnector@427a92e8{HTTP/1.1,[http/1.1]}{127.0.0.1:41882}
   [junit4]   2> 21229 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.e.j.s.Server Started @27252ms
   [junit4]   2> 21230 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/is_s, hostPort=41882, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/cores}
   [junit4]   2> 21240 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 21257 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001'
   [junit4]   2> 21261 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 21261 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 21262 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 21264 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 21285 INFO  (zkCallback-3-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d6a390b name:ZooKeeperConnection Watcher:127.0.0.1:36355/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21285 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 21286 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 21291 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x153d08c96a30002, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 21298 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 21300 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/solr.xml
   [junit4]   2> 21397 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/cores
   [junit4]   2> 21417 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.CoreContainer New CoreContainer 1018070149
   [junit4]   2> 21417 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001]
   [junit4]   2> 21418 WARN  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/lib
   [junit4]   2> 21445 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 21792 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 21795 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 21796 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 21797 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36355/solr
   [junit4]   2> 21809 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 21810 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 21815 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 21841 INFO  (zkCallback-6-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e9f93d4 name:ZooKeeperConnection Watcher:127.0.0.1:36355 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21847 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 21847 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 21891 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 21900 INFO  (zkCallback-7-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b274571 name:ZooKeeperConnection Watcher:127.0.0.1:36355/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21900 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 21906 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 21912 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 21917 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 21930 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 21934 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 21943 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 21945 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 21948 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 21949 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 21950 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 21971 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 21982 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 21984 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 22015 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 22022 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95649319346176004-127.0.0.1:41882_is_s-n_0000000000
   [junit4]   2> 22024 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41882_is_s
   [junit4]   2> 22024 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 22025 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.Overseer Overseer (id=95649319346176004-127.0.0.1:41882_is_s-n_0000000000) starting
   [junit4]   2> 22030 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 22061 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 22063 INFO  (OverseerCollectionConfigSetProcessor-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 22068 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41882_is_s
   [junit4]   2> 22071 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41882_is_s
   [junit4]   2> 22079 INFO  (zkCallback-7-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 22084 INFO  (zkCallback-7-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 22238 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 22238 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 22274 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/cores
   [junit4]   2> 22281 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 22281 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/cores/collection1
   [junit4]   2> 22281 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 22296 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 22296 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [n:127.0.0.1:41882_is_s    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 22299 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 22299 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 22304 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 22339 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 22365 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 22368 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41882/is_s",
   [junit4]   2>   "node_name":"127.0.0.1:41882_is_s",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 22383 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41882/is_s",
   [junit4]   2>   "node_name":"127.0.0.1:41882_is_s",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 22396 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 22410 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 22419 INFO  (zkCallback-7-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 23305 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 23305 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 23306 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 23306 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 23308 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 23309 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/cores/collection1'
   [junit4]   2> 23312 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 23312 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 23317 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 23370 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 23416 WARN  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 23429 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 23750 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 24158 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 25530 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 25546 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 25548 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 25986 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 25989 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 25991 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 25993 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 25996 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 25997 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 26029 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 26038 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:39532/solr_hdfs_home
   [junit4]   2> 26038 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 26047 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/control-001/cores/collection1], dataDir=[hdfs://localhost:39532/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 26063 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3212e247
   [junit4]   2> 26112 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:39532/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 26490 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:39532/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 26508 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:39532/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 26731 WARN  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:39532/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 26812 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:39532/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 26861 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 26861 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 26861 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
   [junit4]   2> 30093 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 30102 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:39532/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 30105 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@37755308
   [junit4]   2> 30955 INFO  (IPC Server handler 4 on 39532) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60265 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-090bac70-9130-4c20-8787-e8b3a2112c31:NORMAL:127.0.0.1:44610|RBW], ReplicaUnderConstruction[[DISK]DS-f8b4d8ce-2678-4169-9a1b-0086cce9b41f:NORMAL:127.0.0.1:60265|RBW]]} size 71
   [junit4]   2> 30956 INFO  (IPC Server handler 6 on 39532) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:44610 is added to blk_1073741825_1001 size 71
   [junit4]   2> 31415 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:39532/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@709dc75c); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 31417 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 31462 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:39532/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 32078 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 32113 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 32128 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 32129 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 32129 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 32129 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 32393 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 32394 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 32394 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 32400 WARN  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 32431 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 32441 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 32442 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 32443 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 32664 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 32665 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 32688 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 32688 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 32688 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 32725 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 32725 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 32732 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=4, maxMergeAtOnceExplicit=7, maxMergedSegmentMB=62.291015625, floorSegmentMB=0.744140625, forceMergeDeletesPctAllowed=14.524766021452072, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7651711111182025
   [junit4]   2> 32876 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:39532/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@709dc75c); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 32876 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 32910 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@633942cd[collection1] main]
   [junit4]   2> 32943 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 32957 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 32957 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 32959 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 32959 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 32995 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 32996 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 32996 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 32996 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 33049 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 33136 INFO  (searcherExecutor-9-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@633942cd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 33146 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 33147 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 33147 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 33148 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1530389123037134848
   [junit4]   2> 33161 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 13.0ms to seed version buckets with highest version 1530389123037134848
   [junit4]   2> 33276 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 33279 INFO  (coreLoadExecutor-8-thread-1-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 33283 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:41882/is_s collection:control_collection shard:shard1
   [junit4]   2> 33292 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 33325 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95649319346176004-core_node1-n_0000000000
   [junit4]   2> 33325 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 33327 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 33327 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 33328 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41882/is_s/collection1/
   [junit4]   2> 33328 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 33328 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:41882/is_s/collection1/ has no replicas
   [junit4]   2> 33329 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 33336 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 33348 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95649319346176004-core_node1-n_0000000000
   [junit4]   2> 33355 INFO  (zkCallback-7-thread-2-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 33366 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:41882/is_s",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 33373 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41882/is_s/collection1/ shard1
   [junit4]   2> 33477 INFO  (zkCallback-7-thread-2-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 33526 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:41882/is_s/collection1/ and leader is http://127.0.0.1:41882/is_s/collection1/
   [junit4]   2> 33527 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41882/is_s
   [junit4]   2> 33527 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 33527 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 33527 INFO  (coreZkRegister-2-thread-1-processing-n:127.0.0.1:41882_is_s x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41882_is_s c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 33530 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41882/is_s",
   [junit4]   2>   "node_name":"127.0.0.1:41882_is_s",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 33531 INFO  (OverseerStateUpdate-95649319346176004-127.0.0.1:41882_is_s-n_0000000000) [n:127.0.0.1:41882_is_s    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41882/is_s",
   [junit4]   2>   "node_name":"127.0.0.1:41882_is_s",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 33565 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 33579 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 33607 INFO  (zkCallback-9-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ae28b2e name:ZooKeeperConnection Watcher:127.0.0.1:36355/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 33607 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 33608 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 33608 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 33812 INFO  (zkCallback-9-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 33812 INFO  (zkCallback-7-thread-2-processing-n:127.0.0.1:41882_is_s) [n:127.0.0.1:41882_is_s    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 33822 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 33833 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 33956 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/shard-1-001/cores/collection1
   [junit4]   2> 33957 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/shard-1-001
   [junit4]   2> 33960 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 34003 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@600afa4c{/is_s,null,AVAILABLE}
   [junit4]   2> 34004 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.e.j.s.ServerConnector Started ServerConnector@33c4f01d{HTTP/1.1,[http/1.1]}{127.0.0.1:54147}
   [junit4]   2> 34004 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.e.j.s.Server Started @40027ms
   [junit4]   2> 34004 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/is_s, hostPort=54147, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/shard-1-001/cores}
   [junit4]   2> 34004 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 34004 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_509C900EC7537C02-001/shard-1-001'
   [junit4]   2> 34005 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 34005 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 34005 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 34042 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 34099 INFO  (zkCallback-10-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3273a50b name:ZooKeeperConnection Watcher:127.0.0.1:36355/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 34099 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 34100 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 34130 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 34130 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[509C900EC7537C02]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.

[...truncated too long message...]

CachingDirectoryFactory.java:351)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:386)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:270)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:387)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:275)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:386)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:270)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:387)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:275)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:386)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:270)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:387)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:275)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:386)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:270)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:387)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:275)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:386)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:270)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:387)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:275)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:386)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:270)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:387)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:275)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:386)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:270)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:387)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:275)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Lucene60, sim=ClassicSimilarity, locale=de-AT, timezone=Canada/Saskatchewan
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=273077912,total=521142272
   [junit4]   2> NOTE: All tests run in this JVM: [SharedFSAutoReplicaFailoverTest, AssignTest, SuggesterTSTTest, TestComplexPhraseQParserPlugin, AnalyticsMergeStrategyTest, CollectionsAPISolrJTest, SpatialHeatmapFacetsTest, DirectSolrConnectionTest, NoCacheHeaderTest, DirectUpdateHandlerOptimizeTest, TestCollectionAPI, FieldMutatingUpdateProcessorTest, TestReversedWildcardFilterFactory, TestFieldTypeCollectionResource, FieldAnalysisRequestHandlerTest, OverseerTest, TestClusterStateMutator, TestNamedUpdateProcessors, TlogReplayBufferedWhileIndexingTest, TestJmxMonitoredMap, SuggesterWFSTTest, TestDefaultStatsCache, DistribJoinFromCollectionTest, TestDynamicLoading, SliceStateTest, TestJmxIntegration, PluginInfoTest, TestFuzzyAnalyzedSuggestions, HdfsChaosMonkeySafeLeaderTest, TestOrdValues, TestXIncludeConfig, TestStressReorder, TestTrackingShardHandlerFactory, TestFastWriter, TestFileDictionaryLookup, QueryParsingTest, LeaderElectionTest, VersionInfoTest, TestConfigReload, BadIndexSchemaTest, UpdateParamsTest, TestConfigSetsAPIZkFailure, TestSolrJ, TestSolrIndexConfig, TestDownShardTolerantSearch, TestPerFieldSimilarityClassic, SmileWriterTest, BlockCacheTest, BadComponentTest, AnalysisErrorHandlingTest, TestAddFieldRealTimeGet, RegexBoostProcessorTest, TestExactStatsCache, HdfsUnloadDistributedZkTest, DebugComponentTest, TestReload, HdfsDirectoryTest, DocValuesMultiTest, UnloadDistributedZkTest, TestCSVResponseWriter, TestJettySolrRunner, SimpleFacetsTest, BadCopyFieldTest, HdfsSyncSliceTest, TestIndexSearcher, TestSchemaResource, IgnoreCommitOptimizeUpdateProcessorFactoryTest, CollectionStateFormat2Test, TestSolrXml, HardAutoCommitTest, MultiThreadedOCPTest, TestCoreDiscovery, TestClassicSimilarityFactory, TestSuggestSpellingConverter, TestChildDocTransformer, TestRebalanceLeaders, SearchHandlerTest, ReplicationFactorTest, ClusterStateTest, StatelessScriptUpdateProcessorFactoryTest, BigEndianAscendingWordSerializerTest, WrapperMergePolicyFactoryTest, TestCloudSchemaless, SSLMigrationTest, TestReloadDeadlock, ScriptEngineTest, OverseerCollectionConfigSetProcessorTest, ShowFileRequestHandlerTest, TestElisionMultitermQuery, PKIAuthenticationIntegrationTest, TestRequestStatusCollectionAPI, TestStandardQParsers, TestCollationField, CachingDirectoryFactoryTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, OpenCloseCoreStressTest, TestReplicationHandler]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandler -Dtests.seed=509C900EC7537C02 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=de-AT -Dtests.timezone=Canada/Saskatchewan -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J2 | TestReplicationHandler (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 9 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([509C900EC7537C02]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:248)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [441/585 (5!)] on J2 in 1070.73s, 13 tests, 1 failure <<< FAILURES!

[...truncated 491 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: There were test failures: 585 suites (10 ignored), 2360 tests, 1 suite-level error, 2 errors, 2 failures, 31 ignored (16 assumptions) [seed: 509C900EC7537C02]

Total time: 315 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




Mime
View raw message