lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 812 - Still Failing
Date Sun, 04 Oct 2015 10:23:02 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/812/

6 tests failed.
FAILED:  org.apache.solr.cloud.CdcrReplicationDistributedZkTest.doTests

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([ED1D3F5775267A70:E57D4A7B7A28527B]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertTrue(Assert.java:54)
	at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.doTestTargetCollectionNotAvailable(CdcrReplicationDistributedZkTest.java:140)
	at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.doTests(CdcrReplicationDistributedZkTest.java:48)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
ERROR: SolrIndexSearcher opens=51 closes=50

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=51 closes=50
	at __randomizedtesting.SeedInfo.seed([ED1D3F5775267A70]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:467)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:233)
	at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
file handle leaks: [FileChannel(/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_ED1D3F5775267A70-001/index-MMapDirectory-002/write.lock)]

Stack Trace:
java.lang.RuntimeException: file handle leaks: [FileChannel(/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_ED1D3F5775267A70-001/index-MMapDirectory-002/write.lock)]
	at org.apache.lucene.mockfile.LeakFS.onClose(LeakFS.java:64)
	at org.apache.lucene.mockfile.FilterFileSystem.close(FilterFileSystem.java:78)
	at org.apache.lucene.mockfile.FilterFileSystem.close(FilterFileSystem.java:79)
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:230)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception
	at org.apache.lucene.mockfile.LeakFS.onOpen(LeakFS.java:47)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:198)
	at org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:167)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:335)
	at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:113)
	at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)
	at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
	at org.apache.lucene.store.FilterDirectory.obtainLock(FilterDirectory.java:94)
	at org.apache.lucene.store.MockDirectoryWrapper.obtainLock(MockDirectoryWrapper.java:985)
	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:761)
	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:78)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:65)
	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:231)
	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:105)
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1642)
	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1784)
	at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:927)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:804)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:672)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:729)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:449)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:440)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:     1) Thread[id=51871, name=searcherExecutor-1783-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 
   1) Thread[id=51871, name=searcherExecutor-1783-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([ED1D3F5775267A70]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=51871, name=searcherExecutor-1783-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=51871, name=searcherExecutor-1783-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([ED1D3F5775267A70]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=4930, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4930, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44311: collection already exists: awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([ED1D3F5775267A70]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 9941 lines...]
   [junit4] Suite: org.apache.solr.cloud.CdcrReplicationDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/init-core-data-001
   [junit4]   2> 525616 INFO  (SUITE-CdcrReplicationDistributedZkTest-seed#[ED1D3F5775267A70]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 525616 INFO  (SUITE-CdcrReplicationDistributedZkTest-seed#[ED1D3F5775267A70]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 525626 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 525627 INFO  (Thread-48053) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 525627 INFO  (Thread-48053) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 525727 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.ZkTestServer start zk server on port:53610
   [junit4]   2> 525727 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 525737 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 525746 INFO  (zkCallback-163-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6ee8debc name:ZooKeeperConnection Watcher:127.0.0.1:53610 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 525747 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 525747 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 525747 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 525775 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 525784 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 525817 INFO  (zkCallback-164-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5659f4f8 name:ZooKeeperConnection Watcher:127.0.0.1:53610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 525817 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 525817 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 525817 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 525819 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 525820 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 525821 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 525822 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 525822 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 525824 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 525824 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 525825 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 525826 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 525827 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 525827 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 525828 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 525828 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 525835 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 525835 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 525836 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 525836 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 525838 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 525838 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 525839 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 525839 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 525841 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 525841 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 525842 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 525842 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 525844 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTests
   [junit4]   2> 526917 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1
   [junit4]   2> 526919 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 526921 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1544f0b2{/,null,AVAILABLE}
   [junit4]   2> 526921 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.e.j.s.ServerConnector Started ServerConnector@669acdc2{HTTP/1.1}{127.0.0.1:47319}
   [junit4]   2> 526921 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.e.j.s.Server Started @530308ms
   [junit4]   2> 526921 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard1, hostContext=/, hostPort=47319, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores}
   [junit4]   2> 526921 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 526922 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/'
   [junit4]   2> 526984 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 526987 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 527015 INFO  (zkCallback-165-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@49d33a3c name:ZooKeeperConnection Watcher:127.0.0.1:53610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 527015 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 527016 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 527024 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 527024 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/solr.xml
   [junit4]   2> 527030 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores
   [junit4]   2> 527030 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.CoreContainer New CoreContainer 1017803995
   [junit4]   2> 527030 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/]
   [junit4]   2> 527030 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/lib
   [junit4]   2> 527030 WARN  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/lib).
   [junit4]   2> 527051 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 527053 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 527053 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 527053 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 527053 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53610/solr
   [junit4]   2> 527053 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 527054 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 527075 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 527075 INFO  (zkCallback-167-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50d843cb name:ZooKeeperConnection Watcher:127.0.0.1:53610 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 527076 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 527076 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 527095 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 527105 INFO  (zkCallback-168-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@683e39a2 name:ZooKeeperConnection Watcher:127.0.0.1:53610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 527106 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 527107 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 527109 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 527112 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 527117 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 527120 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 527122 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 527124 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 527125 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 527129 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 527130 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47319_
   [junit4]   2> 527130 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47319_
   [junit4]   2> 527132 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 527133 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 527135 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 527135 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94630875198652420-127.0.0.1:47319_-n_0000000000
   [junit4]   2> 527136 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47319_
   [junit4]   2> 527136 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 527138 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.Overseer Overseer (id=94630875198652420-127.0.0.1:47319_-n_0000000000) starting
   [junit4]   2> 527140 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 527181 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 527191 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 527192 INFO  (OverseerCollectionConfigSetProcessor-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 527197 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 527246 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 527246 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 527247 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores
   [junit4]   2> 527248 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1, collection=tmp_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 527248 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/
   [junit4]   2> 527248 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 527272 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 527272 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 527272 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 527272 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:47319_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 527273 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47319",
   [junit4]   2>   "node_name":"127.0.0.1:47319_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 527274 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 527274 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47319",
   [junit4]   2>   "node_name":"127.0.0.1:47319_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 527274 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.o.ClusterStateMutator building a new cName: tmp_collection
   [junit4]   2> 527274 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 527276 INFO  (zkCallback-168-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 528274 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 528274 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:tmp_collection
   [junit4]   2> 528274 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Creating collection in ZooKeeper:tmp_collection
   [junit4]   2> 528275 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Looking for collection configName
   [junit4]   2> 528275 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Only one config set found in zk - using it:conf1
   [junit4]   2> 528275 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection
   [junit4]   2> 528276 INFO  (zkCallback-168-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_    ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
   [junit4]   2> 528277 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/tmp_collection
   [junit4]   2> 528278 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 528278 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/'
   [junit4]   2> 528312 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 528330 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 528358 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 528383 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 528404 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 528420 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 528620 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 528621 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 528634 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 528637 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 528705 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 528705 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection tmp_collection
   [junit4]   2> 528705 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 528705 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 528705 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d682ce8
   [junit4]   2> 528706 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/data
   [junit4]   2> 528716 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/data/index/
   [junit4]   2> 528716 WARN  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 528716 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/data/index
   [junit4]   2> 528717 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=578439501, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 528717 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2a6ffa40 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@199080f5),segFN=segments_1,generation=1}
   [junit4]   2> 528717 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 528724 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 528726 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/data/
   [junit4]   2> 528726 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 528749 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 528755 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 528756 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 528758 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 528759 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 528772 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/cdcr,/update,/admin/file
   [junit4]   2> 528773 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 528785 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 528785 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 528786 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 528786 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 528786 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=31, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6272903899804485]
   [junit4]   2> 528787 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2a6ffa40 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@199080f5),segFN=segments_1,generation=1}
   [junit4]   2> 528787 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 528787 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@587a754b[collection1] main
   [junit4]   2> 528787 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/tmp_collection
   [junit4]   2> 528788 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 528788 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 528789 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 528789 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 528789 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 528789 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 528789 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 528789 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 528790 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 528806 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection/cdcr/state
   [junit4]   2> 528813 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrBufferStateManager Created znode /collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 528826 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrProcessStateManager Created znode /collections/tmp_collection/cdcr/state/process
   [junit4]   2> 528914 INFO  (searcherExecutor-587-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@587a754b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 528914 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 528914 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 528914 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 528914 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514094006624059392
   [junit4]   2> 528916 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1514094006624059392
   [junit4]   2> 528916 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 528917 INFO  (coreLoadExecutor-586-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 529047 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:47319 collection:tmp_collection shard:shard2
   [junit4]   2> 529048 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection/leader_elect/shard2/election
   [junit4]   2> 529050 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/tmp_collection/leader_elect/shard2/election/94630875198652420-core_node1-n_0000000000
   [junit4]   2> 529051 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard2
   [junit4]   2> 529052 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 529052 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 529052 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:47319/collection1/
   [junit4]   2> 529052 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 529052 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:47319/collection1/ has no replicas
   [junit4]   2> 529052 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:47319/collection1/ shard2
   [junit4]   2> 529053 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection"} current state version: 1
   [junit4]   2> 529053 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection/leaders/shard2
   [junit4]   2> 529054 INFO  (zkCallback-168-thread-1-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 529058 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 529064 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47319",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 529073 INFO  (zkCallback-168-thread-3-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_    ] o.a.s.h.CdcrProcessStateManager Received new leader state @ tmp_collection:shard2
   [junit4]   2> 529173 INFO  (zkCallback-168-thread-3-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 529211 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:47319/collection1/ and leader is http://127.0.0.1:47319/collection1/
   [junit4]   2> 529211 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47319
   [junit4]   2> 529211 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 529211 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 529211 INFO  (coreZkRegister-580-thread-1-processing-n:127.0.0.1:47319_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:47319_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 529213 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47319",
   [junit4]   2>   "node_name":"127.0.0.1:47319_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 529214 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47319",
   [junit4]   2>   "node_name":"127.0.0.1:47319_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 529319 INFO  (zkCallback-168-thread-3-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 529973 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/cores/collection1
   [junit4]   2> 529987 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 529993 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c7ef56a{/,null,AVAILABLE}
   [junit4]   2> 529993 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.e.j.s.ServerConnector Started ServerConnector@33608512{HTTP/1.1}{127.0.0.1:56153}
   [junit4]   2> 529993 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.e.j.s.Server Started @533381ms
   [junit4]   2> 529993 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard2, hostContext=/, hostPort=56153, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/cores}
   [junit4]   2> 529994 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 529994 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/'
   [junit4]   2> 530033 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 530046 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 530047 INFO  (zkCallback-169-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ea2cd2e name:ZooKeeperConnection Watcher:127.0.0.1:53610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 530047 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 530048 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 530049 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 530049 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/solr.xml
   [junit4]   2> 530054 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/cores
   [junit4]   2> 530055 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.CoreContainer New CoreContainer 89201711
   [junit4]   2> 530055 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/]
   [junit4]   2> 530055 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/lib
   [junit4]   2> 530055 WARN  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/lib).
   [junit4]   2> 530064 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 530065 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 530066 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 530066 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 530066 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53610/solr
   [junit4]   2> 530066 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 530066 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 530077 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 530081 INFO  (zkCallback-171-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33f7f8b6 name:ZooKeeperConnection Watcher:127.0.0.1:53610 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 530082 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 530082 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 530090 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 530091 INFO  (zkCallback-172-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36f590a6 name:ZooKeeperConnection Watcher:127.0.0.1:53610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 530091 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 530098 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 531106 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56153_
   [junit4]   2> 531107 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56153_
   [junit4]   2> 531108 INFO  (zkCallback-172-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 531108 INFO  (zkCallback-168-thread-3-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 531110 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 531110 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94630875198652423-127.0.0.1:56153_-n_0000000001
   [junit4]   2> 531111 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94630875198652420-127.0.0.1:47319_-n_0000000000 to know if I could be the leader
   [junit4]   2> 531124 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 531125 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 531125 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/cores
   [junit4]   2> 531126 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/cores/collection1, collection=tmp_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 531126 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/cores/collection1/
   [junit4]   2> 531126 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 531128 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 531128 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 531128 INFO  (TEST-CdcrReplicationDistributedZkTest.doTests-seed#[ED1D3F5775267A70]) [n:127.0.0.1:56153_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 531128 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 531129 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Registering watch for collection tmp_collection
   [junit4]   2> 531129 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch tmp_collection
   [junit4]   2> 531130 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56153",
   [junit4]   2>   "node_name":"127.0.0.1:56153_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 531130 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for tmp_collection
   [junit4]   2> 531130 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56153",
   [junit4]   2>   "node_name":"127.0.0.1:56153_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 531130 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 531130 INFO  (OverseerStateUpdate-94630875198652420-127.0.0.1:47319_-n_0000000000) [n:127.0.0.1:47319_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 531131 INFO  (zkCallback-168-thread-3-processing-n:127.0.0.1:47319_) [n:127.0.0.1:47319_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 531131 INFO  (zkCallback-172-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 531133 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 531133 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 531133 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:tmp_collection
   [junit4]   2> 531133 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 531133 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/tmp_collection
   [junit4]   2> 531134 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 531134 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_ED1D3F5775267A70-001/jetty-002/cores/collection1/'
   [junit4]   2> 531150 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 531155 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 531158 INFO  (coreLoadExecutor-598-thread-1-processing-n:127.0.0.1:56153_) [n:127.0.0.1:56153_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 531165 INFO  (cor

[...truncated too long message...]

tocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 9 moreThrowable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4927, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:55095, http://127.0.0.1:47520, http://127.0.0.1:44311, http://127.0.0.1:55604, http://127.0.0.1:40652]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([ED1D3F5775267A70]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:55095, http://127.0.0.1:47520, http://127.0.0.1:44311, http://127.0.0.1:55604, http://127.0.0.1:40652]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:55604
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:587)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:170)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 9 more
   [junit4]   2> 1182581 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[ED1D3F5775267A70]-worker) [n:127.0.0.1:55604_ c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {other_tl1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), _version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), multiDefault=PostingsFormat(name=Memory doPackFST= true), a_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), intDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_i1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_i_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=PostingsFormat(name=Direct), range_facet_l=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), timestamp=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Lucene50)}, sim=ClassicSimilarity, locale=ru_RU, timezone=Asia/Beirut
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=4,threads=1,free=40277344,total=534773760
   [junit4]   2> NOTE: All tests run in this JVM: [TestDynamicLoading, TestRequestStatusCollectionAPI, DistanceUnitsTest, FullHLLTest, AtomicUpdatesTest, DocValuesMissingTest, TestLFUCache, TestUpdate, DirectSolrSpellCheckerTest, TestFileDictionaryLookup, CircularListTest, TestUtils, UniqFieldsUpdateProcessorFactoryTest, ClusterStateUpdateTest, HdfsRecoveryZkTest, TestRebalanceLeaders, LoggingHandlerTest, TestStressVersions, TestManagedSchema, TermsComponentTest, SpatialHeatmapFacetsTest, SimpleMLTQParserTest, TestQuerySenderListener, TestFreeTextSuggestions, SchemaVersionSpecificBehaviorTest, HdfsLockFactoryTest, DistributedQueueTest, DistributedTermsComponentTest, DateRangeFieldTest, TestCoreContainer, HLLSerializationTest, TestConfigSets, TestAnalyzeInfixSuggestions, TestReplicationHandlerBackup, DistributedFacetPivotLongTailTest, SimplePostToolTest, TestJsonRequest, SolrInfoMBeanTest, TestSuggestSpellingConverter, BitVectorTest, DistributedExpandComponentTest, TestRestoreCore, SpatialFilterTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [356/546] on J1 in 681.60s, 1 test, 1 error <<< FAILURES!

[...truncated 602 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:782: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:719: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:516: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1432: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:987: There were test failures: 546 suites (8 ignored), 2145 tests, 4 suite-level errors, 1 error, 1 failure, 1127 ignored (14 assumptions) [seed: ED1D3F5775267A70]

Total time: 150 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Mime
View raw message