lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-master - Build # 974 - Still Failing
Date Fri, 01 Apr 2016 00:23:27 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/974/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
ObjectTracker found 4 object(s) that were not released!!! [SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not released!!! [SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor]
	at __randomizedtesting.SeedInfo.seed([F1F8554AE222677C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:248)
	at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:     1) Thread[id=27929, name=searcherExecutor-6300-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 
   1) Thread[id=27929, name=searcherExecutor-6300-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([F1F8554AE222677C]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=27929, name=searcherExecutor-6300-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=27929, name=searcherExecutor-6300-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([F1F8554AE222677C]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=14573, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=14573, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33532: collection already exists: awholynewstresscollection_collection1_8
	at __randomizedtesting.SeedInfo.seed([F1F8554AE222677C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1165)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:935)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1614)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:970)




Build Log:
[...truncated 12024 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/init-core-data-001
   [junit4]   2> 2392794 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[F1F8554AE222677C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 2392794 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[F1F8554AE222677C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2392795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2392795 INFO  (Thread-7231) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2392795 INFO  (Thread-7231) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2392896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.ZkTestServer start zk server on port:49001
   [junit4]   2> 2392896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2392907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2392923 INFO  (zkCallback-1408-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e2a5d35 name:ZooKeeperConnection Watcher:127.0.0.1:49001 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2392927 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2392927 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2392927 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2392934 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2392935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2392935 INFO  (zkCallback-1409-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e515ffd name:ZooKeeperConnection Watcher:127.0.0.1:49001/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2392935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2392936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2392936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2392937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2392937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2392938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2392939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2392939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2392940 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2392940 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2392941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2392941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2392943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2392943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2392944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2392944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2392944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2392945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2392945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2392946 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2392946 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2392947 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2392948 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2392948 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2392949 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2392949 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2392950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2392950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2392955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2392959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2392983 INFO  (zkCallback-1410-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@49494d86 name:ZooKeeperConnection Watcher:127.0.0.1:49001 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2392983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2392984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2392984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2393039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2393046 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2393063 INFO  (zkCallback-1411-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@65d80c53 name:ZooKeeperConnection Watcher:127.0.0.1:49001/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2393067 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2393067 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2393068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 2393068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 2393123 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 2393124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 2393270 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 2393270 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 2393346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2393346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2393348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 2393348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 2393349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 2393350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 2393351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 2393351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 2393352 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2393352 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2393354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2393354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2393356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 2393356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 2393357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 2393357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 2393358 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 2393358 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 2393416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1
   [junit4]   2> 2393418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2393419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d71c15c{/,null,AVAILABLE}
   [junit4]   2> 2393421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@36a4d142{HTTP/1.1,[http/1.1]}{127.0.0.1:37891}
   [junit4]   2> 2393421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.e.j.s.Server Started @2399284ms
   [junit4]   2> 2393421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/tempDir-001/control/data, hostContext=/, hostPort=37891, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores}
   [junit4]   2> 2393421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2393421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001'
   [junit4]   2> 2393421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2393421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2393421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2393439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2393447 INFO  (zkCallback-1412-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2ac88f0b name:ZooKeeperConnection Watcher:127.0.0.1:49001/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2393451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2393451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2393487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2393487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/solr.xml
   [junit4]   2> 2393491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores
   [junit4]   2> 2393491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.CoreContainer New CoreContainer 817556900
   [junit4]   2> 2393491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001]
   [junit4]   2> 2393491 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/lib
   [junit4]   2> 2393491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2393496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2393497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2393497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2393497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49001/solr
   [junit4]   2> 2393497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2393497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2393507 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2393520 INFO  (zkCallback-1415-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36503f93 name:ZooKeeperConnection Watcher:127.0.0.1:49001 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2393528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2393529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2393542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2393543 INFO  (zkCallback-1416-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10a0c69a name:ZooKeeperConnection Watcher:127.0.0.1:49001/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2393543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2393547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2393548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2393550 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2393551 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2393552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2393553 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2393553 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2393554 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2393555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2393555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2393556 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2393557 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2393557 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2393558 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2393558 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95647718093094918-127.0.0.1:37891_-n_0000000000
   [junit4]   2> 2393559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37891_
   [junit4]   2> 2393559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2393559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.Overseer Overseer (id=95647718093094918-127.0.0.1:37891_-n_0000000000) starting
   [junit4]   2> 2393583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2393646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2393651 INFO  (OverseerCollectionConfigSetProcessor-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2393656 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37891_
   [junit4]   2> 2393657 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2393659 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:37891_
   [junit4]   2> 2393672 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2393675 INFO  (zkCallback-1416-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2393675 INFO  (zkCallback-1416-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 2393696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2393696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2393697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores
   [junit4]   2> 2393697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2393697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1
   [junit4]   2> 2393698 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2393715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2
   [junit4]   2> 2393715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:37891_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2393731 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2393731 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2393783 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:37891",
   [junit4]   2>   "node_name":"127.0.0.1:37891_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2393791 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2393797 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:37891",
   [junit4]   2>   "node_name":"127.0.0.1:37891_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2393811 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2393823 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2393840 INFO  (zkCallback-1416-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2394798 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2394798 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2394799 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2394799 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2394815 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2394815 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1'
   [junit4]   2> 2394815 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2394815 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2394823 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2394847 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2394880 WARN  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2394881 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2394895 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2394911 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2395089 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2395093 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2395094 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2395107 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2395108 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2395116 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2395117 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2395118 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2
   [junit4]   2> 2395118 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2395118 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2395118 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1/data/]
   [junit4]   2> 2395118 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@563c2824
   [junit4]   2> 2395118 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1/data
   [junit4]   2> 2395118 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2395119 WARN  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2395119 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1/data/index
   [junit4]   2> 2395119 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=12, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.16080596704024563]
   [junit4]   2> 2395131 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@57fc7cda lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@54a6c44d),segFN=segments_1,generation=1}
   [junit4]   2> 2395131 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2395140 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1/data/
   [junit4]   2> 2395140 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2395141 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2395145 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2395146 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2395146 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2395146 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2395146 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2395146 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2395146 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2395146 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2395146 WARN  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2395147 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2395148 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2395148 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2395149 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2395156 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 2395156 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2395163 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2395163 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2395164 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2395164 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2395167 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3374291320878623]
   [junit4]   2> 2395167 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@57fc7cda lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@54a6c44d),segFN=segments_1,generation=1}
   [junit4]   2> 2395168 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2395168 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5242d02d[collection1] main]
   [junit4]   2> 2395168 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2395168 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2395168 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2395169 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1530363491975692288
   [junit4]   2> 2395171 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1530363491975692288
   [junit4]   2> 2395171 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2395171 INFO  (searcherExecutor-4248-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5242d02d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2395172 INFO  (coreLoadExecutor-4247-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2395179 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:37891 collection:control_collection shard:shard1
   [junit4]   2> 2395191 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2395201 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95647718093094918-core_node1-n_0000000000
   [junit4]   2> 2395202 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 2395203 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2395203 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2395203 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37891/collection1/
   [junit4]   2> 2395203 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2395203 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:37891/collection1/ has no replicas
   [junit4]   2> 2395203 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2395204 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2395205 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95647718093094918-core_node1-n_0000000000
   [junit4]   2> 2395206 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37891/collection1/ shard1
   [junit4]   2> 2395206 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:37891",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2395248 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395256 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395271 INFO  (zkCallback-1418-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40ac5d30 name:ZooKeeperConnection Watcher:127.0.0.1:49001/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395273 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2395323 INFO  (zkCallback-1418-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 2395324 INFO  (zkCallback-1416-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2395324 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2395331 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2395363 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:37891/collection1/ and leader is http://127.0.0.1:37891/collection1/
   [junit4]   2> 2395363 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37891
   [junit4]   2> 2395363 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2395363 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2395363 INFO  (coreZkRegister-4241-thread-1-processing-n:127.0.0.1:37891_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37891_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2395380 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:37891",
   [junit4]   2>   "node_name":"127.0.0.1:37891_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 2395380 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:37891",
   [junit4]   2>   "node_name":"127.0.0.1:37891_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2395496 INFO  (zkCallback-1416-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2395591 INFO  (zkCallback-1418-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2395621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/shard-1-001/cores/collection1
   [junit4]   2> 2395621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/shard-1-001
   [junit4]   2> 2395622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2395627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@45e5cd74{/,null,AVAILABLE}
   [junit4]   2> 2395628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7ab25a63{HTTP/1.1,[http/1.1]}{127.0.0.1:53667}
   [junit4]   2> 2395628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.e.j.s.Server Started @2401491ms
   [junit4]   2> 2395628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=53667, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/shard-1-001/cores}
   [junit4]   2> 2395628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2395628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/shard-1-001'
   [junit4]   2> 2395628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2395628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2395628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395629 INFO  (zkCallback-1419-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@415c9b80 name:ZooKeeperConnection Watcher:127.0.0.1:49001/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2395630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/shard-1-001/solr.xml
   [junit4]   2> 2395636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/shard-1-001/cores
   [junit4]   2> 2395636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.CoreContainer New CoreContainer 1639405364
   [junit4]   2> 2395636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/shard-1-001]
   [junit4]   2> 2395637 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F1F8554AE222677C-001/shard-1-001/lib
   [junit4]   2> 2395637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2395638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2395638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2395638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2395638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49001/solr
   [junit4]   2> 2395638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2395638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395639 INFO  (zkCallback-1422-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f43ddcb name:ZooKeeperConnection Watcher:127.0.0.1:49001 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395640 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395640 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395642 INFO  (zkCallback-1423-thread-1-processing-n:127.0.0.1:53667_) [n:127.0.0.1:53667_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6606473a name:ZooKeeperConnection Watcher:127.0.0.1:49001/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2395647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2395649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2395649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95647718093094922-127.0.0.1:53667_-n_0000000001
   [junit4]   2> 2395649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95647718093094918-127.0.0.1:37891_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2395649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.ZkController Publish node=127.0.0.1:53667_ as DOWN
   [junit4]   2> 2395650 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:53667_"} current state version: 3
   [junit4]   2> 2395650 INFO  (OverseerStateUpdate-95647718093094918-127.0.0.1:37891_-n_0000000000) [n:127.0.0.1:37891_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:53667_
   [junit4]   2> 2395763 INFO  (zkCallback-1418-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2395764 INFO  (zkCallback-1416-thread-1-processing-n:127.0.0.1:37891_) [n:127.0.0.1:37891_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2395764 INFO  (zkCallback-1423-thread-1-processing-n:127.0.0.1:53667_) [n:127.0.0.1:53667_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2396651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53667_
   [junit4]   2> 2396745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F1F8554AE222677C]) [n:127.0.0.1:53667_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53667_
   [junit4]   2> 2396755 INFO  (zkCallback-1418-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2396756 INFO  (zkCallback-1418-thread-1) [    ] 

[...truncated too long message...]

ed from SUITE scope at org.apache.solr.core.TestLazyCores: 
   [junit4]   2>    1) Thread[id=27929, name=searcherExecutor-6300-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Apr 01, 2016 12:10:11 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=27929, name=searcherExecutor-6300-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]   2> Apr 01, 2016 12:10:14 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=27929, name=searcherExecutor-6300-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene60): {v_t=FSTOrd50, id=FSTOrd50, v_s=Lucene50(blocksize=128), v_f=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{}, maxPointsInLeafNode=969, maxMBSortInHeap=5.627296327161023, sim=ClassicSimilarity, locale=da-DK, timezone=Pacific/Galapagos
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=2,free=217232080,total=509083648
   [junit4]   2> NOTE: All tests run in this JVM: [ZkSolrClientTest, HdfsRestartWhileUpdatingTest, TestConfigSetsAPIExclusivity, TestFieldSortValues, TestSimpleTrackingShardHandler, TermVectorComponentTest, DistributedExpandComponentTest, TestLMJelinekMercerSimilarityFactory, TestSolrConfigHandlerCloud, TestBM25SimilarityFactory, TestQueryTypes, OpenExchangeRatesOrgProviderTest, TestTrie, SoftAutoCommitTest, CSVRequestHandlerTest, TestFiltering, TestCoreContainer, TestSolrCoreProperties, SuggesterFSTTest, TestReqParamsAPI, NotRequiredUniqueKeyTest, TestUpdate, TestSchemaVersionResource, TestRequestForwarding, TestNoOpRegenerator, TestMiniSolrCloudClusterBase, ExitableDirectoryReaderTest, SimpleCollectionCreateDeleteTest, DistributedTermsComponentTest, HdfsChaosMonkeyNothingIsSafeTest, TestScoreJoinQPScore, TestPartialUpdateDeduplication, ShardSplitTest, ZkControllerTest, DocumentBuilderTest, TestWordDelimiterFilterFactory, TestSystemIdResolver, TestSchemaNameResource, TestMinMaxOnMultiValuedField, MultiTermTest, TestUseDocValuesAsStored2, RollingRestartTest, TestCursorMarkWithoutUniqueKey, OverriddenZkACLAndCredentialsProvidersTest, DistributedIntervalFacetingTest, TestSolrQueryResponse, TestFastLRUCache, VersionInfoTest, ClassificationUpdateProcessorFactoryTest, TestRecoveryHdfs, TestExactStatsCache, ChaosMonkeyNothingIsSafeTest, DirectSolrSpellCheckerTest, TestRuleBasedAuthorizationPlugin, AnalysisErrorHandlingTest, FullHLLTest, MigrateRouteKeyTest, TestConfigOverlay, DistributedVersionInfoTest, TestCodecSupport, TestAuthenticationFramework, ConfigSetsAPITest, CollectionsAPIAsyncDistributedZkTest, TestRandomRequestDistribution, SpatialRPTFieldTypeTest, TestSurroundQueryParser, LoggingHandlerTest, SecurityConfHandlerTest, TestInfoStreamLogging, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DistanceFunctionTest, FileUtilsTest, AliasIntegrationTest, ConvertedLegacyTest, TestQuerySenderListener, HdfsNNFailoverTest, TestSolrDynamicMBean, DateRangeFieldTest, SimplePostToolTest, DOMUtilTest, WordBreakSolrSpellCheckerTest, BigEndianAscendingWordSerializerTest, RulesTest, TestStressVersions, TestSolrFieldCacheMBean, AsyncMigrateRouteKeyTest, TestNonDefinedSimilarityFactory, MBeansHandlerTest, TestCollapseQParserPlugin, SparseHLLTest, DeleteShardTest, TestBinaryResponseWriter, SpellingQueryConverterTest, OpenCloseCoreStressTest, TestHashPartitioner, TestRangeQuery, StatsComponentTest, TestFunctionQuery, TestLazyCores]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLazyCores -Dtests.seed=F1F8554AE222677C -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=da-DK -Dtests.timezone=Pacific/Galapagos -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | TestLazyCores (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 4 object(s) that were not released!!! [SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F1F8554AE222677C]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:248)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 
   [junit4]    >    1) Thread[id=27929, name=searcherExecutor-6300-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F1F8554AE222677C]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=27929, name=searcherExecutor-6300-thread-1, state=WAITING, group=TGRP-TestLazyCores]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F1F8554AE222677C]:0)
   [junit4] Completed [370/584 (2!)] on J1 in 152.21s, 8 tests, 1 failure, 2 errors <<< FAILURES!

[...truncated 654 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/lucene/common-build.xml:1014: There were test failures: 584 suites (10 ignored), 2353 tests, 3 suite-level errors, 1 error, 699 ignored (13 assumptions) [seed: F1F8554AE222677C]

Total time: 263 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




Mime
View raw message