lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 746 - Still Failing
Date Sat, 25 Jul 2015 07:38:36 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/746/

2 tests failed.
REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
[replication.properties, index.20150725130634765, index.properties, index.20150725130630315] expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: [replication.properties, index.20150725130634765, index.properties, index.20150725130630315] expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([EC5FBD1647494C2F:37F4BDD04261259C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.handler.TestReplicationHandler.checkForSingleIndex(TestReplicationHandler.java:818)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:785)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=4348, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4348, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58453: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([EC5FBD1647494C2F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10411 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/init-core-data-001
   [junit4]   2> 322553 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 322553 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 322598 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 322612 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 322616 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 322633 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_34374_hdfs____.vgqltz/webapp
   [junit4]   2> 322757 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 323188 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34374
   [junit4]   2> 323265 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 323267 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 323288 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36027_datanode____vgzk7t/webapp
   [junit4]   2> 323394 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 323785 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36027
   [junit4]   2> 323847 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 323851 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 323875 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47758_datanode____.n3cste/webapp
   [junit4]   2> 323932 INFO  (IPC Server handler 4 on 41535) [    ] BlockStateChange BLOCK* processReport: from storage DS-3bb541e2-abde-4324-9705-e51f67e84540 node DatanodeRegistration(127.0.0.1, datanodeUuid=dd7f15c4-e6f9-4d65-adc7-a977ac02e0d8, infoPort=36027, ipcPort=49972, storageInfo=lv=-56;cid=testClusterID;nsid=257256401;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 323933 INFO  (IPC Server handler 4 on 41535) [    ] BlockStateChange BLOCK* processReport: from storage DS-2c7c6212-04cf-4343-a4ed-f571c045f969 node DatanodeRegistration(127.0.0.1, datanodeUuid=dd7f15c4-e6f9-4d65-adc7-a977ac02e0d8, infoPort=36027, ipcPort=49972, storageInfo=lv=-56;cid=testClusterID;nsid=257256401;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 324002 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 325470 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47758
   [junit4]   2> 325583 INFO  (IPC Server handler 8 on 41535) [    ] BlockStateChange BLOCK* processReport: from storage DS-7d9016e2-4e03-40cf-a44a-a294e530f1e7 node DatanodeRegistration(127.0.0.1, datanodeUuid=7b0dac37-5002-4781-a663-70d0ed4562f0, infoPort=47758, ipcPort=48403, storageInfo=lv=-56;cid=testClusterID;nsid=257256401;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 325585 INFO  (IPC Server handler 8 on 41535) [    ] BlockStateChange BLOCK* processReport: from storage DS-09b68d51-a376-4be0-9ee6-b491bd109ec5 node DatanodeRegistration(127.0.0.1, datanodeUuid=7b0dac37-5002-4781-a663-70d0ed4562f0, infoPort=47758, ipcPort=48403, storageInfo=lv=-56;cid=testClusterID;nsid=257256401;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 325646 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 325647 INFO  (Thread-1875) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 325647 INFO  (Thread-1875) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 325747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkTestServer start zk server on port:54141
   [junit4]   2> 325747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325748 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325757 INFO  (zkCallback-337-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@170957b1 name:ZooKeeperConnection Watcher:127.0.0.1:54141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 325760 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325765 INFO  (zkCallback-338-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@cb8bb44 name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 325767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 325768 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 325770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 325771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 325771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 325774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 325774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 325776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 325776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 325777 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 325778 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 325779 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 325779 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 325781 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 325781 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 325783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 325783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 325784 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 325785 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 325786 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 325786 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 325788 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 325788 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 325790 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 325790 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 325792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325797 INFO  (zkCallback-339-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3162afa4 name:ZooKeeperConnection Watcher:127.0.0.1:54141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325797 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 325800 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325801 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325803 INFO  (zkCallback-340-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@12c34ec9 name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325803 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325803 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325804 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 325804 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 325806 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 325806 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 325807 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 325808 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 325810 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 325810 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 325811 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 325812 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 325821 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 325822 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 325823 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 325823 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 325825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 325825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 325826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 325826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 325828 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 325828 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 325829 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 325830 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 325831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 325831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 326142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1
   [junit4]   2> 326143 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 326145 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4a32de0c{/,null,AVAILABLE}
   [junit4]   2> 326146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@61e4bbe1{HTTP/1.1}{127.0.0.1:58751}
   [junit4]   2> 326146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.Server Started @329696ms
   [junit4]   2> 326146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:41535/hdfs__localhost_41535__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001_tempDir-002_control_data, hostContext=/, hostPort=58751, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores}
   [junit4]   2> 326147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 326147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/'
   [junit4]   2> 326162 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/solr.xml
   [junit4]   2> 326167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores
   [junit4]   2> 326167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1935429587
   [junit4]   2> 326167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/]
   [junit4]   2> 326167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/lib
   [junit4]   2> 326168 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/lib).
   [junit4]   2> 326174 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 326176 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 326176 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 326176 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 326177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54141/solr
   [junit4]   2> 326177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 326177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 326177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 326179 INFO  (zkCallback-342-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75b2e7a9 name:ZooKeeperConnection Watcher:127.0.0.1:54141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 326179 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 326180 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 326181 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 326182 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34dc3f2c name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 326182 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 326184 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 326186 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 326188 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 326190 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 326192 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 326195 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 326197 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 326199 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 326200 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58751_
   [junit4]   2> 326200 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58751_
   [junit4]   2> 326202 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 326203 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 326208 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 326209 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94228049315233797-127.0.0.1:58751_-n_0000000000
   [junit4]   2> 326210 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58751_
   [junit4]   2> 326210 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 326212 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer Overseer (id=94228049315233797-127.0.0.1:58751_-n_0000000000) starting
   [junit4]   2> 326214 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 326220 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 326221 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 326221 INFO  (OverseerCollectionProcessor-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 326221 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 326224 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 326225 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 326225 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores
   [junit4]   2> 326226 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 326226 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1/
   [junit4]   2> 326227 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 326227 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 326227 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 326228 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 326228 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 326228 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 326228 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 326229 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "node_name":"127.0.0.1:58751_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 326229 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "node_name":"127.0.0.1:58751_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 326230 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 326230 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 326237 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 326239 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 327228 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 327229 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 327229 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 327229 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 327230 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 327230 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1/'
   [junit4]   2> 327254 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 327260 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 327263 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 327271 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 327275 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 327281 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 327372 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 327378 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 327379 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 327390 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 327393 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 327395 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 327395 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 327395 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 327396 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 327396 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 327396 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 327397 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 327398 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 327398 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 327408 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:41535/solr_hdfs_home
   [junit4]   2> 327409 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 327409 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 327409 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4e98902a
   [junit4]   2> 327410 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 327435 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 327435 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 327440 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 327454 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 327454 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 327468 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 327468 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 327469 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=6, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 327754 INFO  (IPC Server handler 8 on 41535) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42510 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-09b68d51-a376-4be0-9ee6-b491bd109ec5:NORMAL:127.0.0.1:42510|RBW], ReplicaUnderConstruction[[DISK]DS-3bb541e2-abde-4324-9705-e51f67e84540:NORMAL:127.0.0.1:44118|RBW]]} size 71
   [junit4]   2> 327757 INFO  (IPC Server handler 9 on 41535) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:44118 is added to blk_1073741825_1001 size 71
   [junit4]   2> 328154 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6fc8a19 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@734f3b2c; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 328154 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 328157 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 328161 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 328161 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 328163 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 328165 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 328166 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 328167 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 328168 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 328171 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 328172 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 328179 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 328179 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 328195 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 328195 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 328198 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=74.4296875, floorSegmentMB=0.7265625, forceMergeDeletesPctAllowed=21.66815875061856, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 328289 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6fc8a19 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@734f3b2c; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 328289 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 328290 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@c263547[collection1] main
   [junit4]   2> 328290 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 328291 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 328291 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 328291 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 328292 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 328292 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 328292 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 328293 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 328293 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 328293 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 328295 INFO  (searcherExecutor-1118-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@c263547[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1507648791828758528
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0 ms to seed version buckets with highest version 1507648791828758528
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 328305 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 328305 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:58751 collection:control_collection shard:shard1
   [junit4]   2> 328306 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 328312 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94228049315233797-core_node1-n_0000000000
   [junit4]   2> 328313 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 328314 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 328314 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 328314 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 328315 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58751/collection1/
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:58751/collection1/ has no replicas
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58751/collection1/ shard1
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 328316 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 328317 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 328319 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 328319 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 328422 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 328423 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 328470 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:58751/collection1/ and leader is http://127.0.0.1:58751/collection1/
   [junit4]   2> 328470 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58751
   [junit4]   2> 328470 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 328470 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 328471 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 328473 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 328473 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "node_name":"127.0.0.1:58751_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 328474 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "node_name":"127.0.0.1:58751_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 328576 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 328578 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 328730 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 328731 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 328732 INFO  (zkCallback-345-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5796dc1c name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 328733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 328733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 328733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 328737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 329054 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores/collection1
   [junit4]   2> 329055 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001
   [junit4]   2> 329056 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 329058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34c1bbe{/,null,AVAILABLE}
   [junit4]   2> 329058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@24af1712{HTTP/1.1}{127.0.0.1:38533}
   [junit4]   2> 329058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.Server Started @332608ms
   [junit4]   2> 329058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:41535/hdfs__localhost_41535__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=38533, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores}
   [junit4]   2> 329059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 329059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/'
   [junit4]   2> 329074 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/solr.xml
   [junit4]   2> 329079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores
   [junit4]   2> 329079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1236840958
   [junit4]   2> 329079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/]
   [junit4]   2> 329079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/lib
   [junit4]   2> 329079 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/lib).
   [junit4]   2> 329086 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 329088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 329088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 329088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 329088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54141/solr
   [junit4]   2> 329089 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 329089 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 329089 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 329091 INFO  (zkCallback-347-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25a149e6 name:ZooKeeperConnection Watcher:127.0.0.1:54141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 329091 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 329091 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 329095 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 329096 INFO  (zkCallback-348-thread-1-processing-n:127.0.0.1:38533_) [n:127.0.0.1:38533_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f7381c3 name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 329096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 329100 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 330104 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38533_
   [junit4]   2> 330105 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38533_
   [junit4]   2> 330108 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 330109 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94228049315233800-127.0.0.1:38533_-n_0000000001
   [junit4]   2> 330110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94228049315233797-127.0.0.1:58751_-n_0000000000 to know if I could be the leader
   [junit4]   2> 330110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 330111 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 330111 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores
   [junit4]   2> 330112 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 330112 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) 

[...truncated too long message...]

.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data;done=false>>]
   [junit4]   2> 2840731 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data
   [junit4]   2> 2840732 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data/index;done=false>>]
   [junit4]   2> 2840732 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data/index
   [junit4]   2> 2840732 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@779e9c4c{/solr,null,UNAVAILABLE}
   [junit4]   2> 2840746 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@5c0dfc2e{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2840747 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1132517580
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2e6851c6
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2840755 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2840755 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2840755 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data;done=false>>]
   [junit4]   2> 2840755 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data
   [junit4]   2> 2840756 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data/index;done=false>>]
   [junit4]   2> 2840756 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data/index
   [junit4]   2> 2840757 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@11890b97{/solr,null,UNAVAILABLE}
   [junit4]   2> 2840758 INFO  (SUITE-TestReplicationHandler-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=ar_IQ, timezone=Asia/Dacca
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=4,threads=1,free=324170384,total=510656512
   [junit4]   2> NOTE: All tests run in this JVM: [ZkNodePropsTest, ConnectionManagerTest, RollingRestartTest, SignatureUpdateProcessorFactoryTest, CustomCollectionTest, DistributedFacetPivotLongTailTest, TestCodecSupport, TestBinaryField, SuggesterFSTTest, ResponseHeaderTest, TestDocBasedVersionConstraints, TestIBSimilarityFactory, TestJettySolrRunner, AssignTest, SolrCoreTest, UniqFieldsUpdateProcessorFactoryTest, TestSchemaNameResource, SmileWriterTest, TestSolrDynamicMBean, BlockDirectoryTest, TestConfig, IndexBasedSpellCheckerTest, TestJmxIntegration, TestRebalanceLeaders, TestUpdate, TestMiniSolrCloudCluster, TestDynamicFieldResource, TestDynamicLoading, ChangedSchemaMergeTest, SharedFSAutoReplicaFailoverUtilsTest, PreAnalyzedUpdateProcessorTest, DataDrivenBlockJoinTest, DocExpirationUpdateProcessorFactoryTest, URLClassifyProcessorTest, TestArbitraryIndexDir, ChaosMonkeyNothingIsSafeTest, TestClusterStateMutator, SimpleCollectionCreateDeleteTest, RemoteQueryErrorTest, TestConfigReload, TestCloudPivotFacet, UUIDFieldTest, DistributedSpellCheckComponentTest, SegmentsInfoRequestHandlerTest, TestSimpleTrackingShardHandler, TestSchemaManager, TestSQLHandler, TestCollapseQParserPlugin, BasicDistributedZkTest, DistributedFacetPivotWhiteBoxTest, PeerSyncTest, SolrRequestParserTest, SolrIndexConfigTest, TestPhraseSuggestions, MinimalSchemaTest, TestChildDocTransformer, TestRequestStatusCollectionAPI, CdcrReplicationHandlerTest, TestRTimer, SuggesterWFSTTest, TermVectorComponentTest, TestManagedSynonymFilterFactory, TestPivotHelperCode, BJQParserTest, TestFaceting, SchemaVersionSpecificBehaviorTest, TestFreeTextSuggestions, TestMergePolicyConfig, TestFileDictionaryLookup, TestClassNameShortening, CollectionTooManyReplicasTest, SyncSliceTest, RecoveryAfterSoftCommitTest, TestStressLucene, TestSolrDeletionPolicy1, TestWordDelimiterFilterFactory, TestJsonFacets, TestRandomFaceting, FastVectorHighlighterTest, QueryElevationComponentTest, VersionInfoTest, TestManagedResource, HLLSerializationTest, TermsComponentTest, CursorMarkTest, BasicDistributedZk2Test, LeaderElectionIntegrationTest, BasicZkTest, TestReplicationHandler]
   [junit4] Completed [276/525] on J0 in 547.05s, 13 tests, 1 failure <<< FAILURES!

[...truncated 778 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:721: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:658: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:517: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1429: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:984: There were test failures: 525 suites, 2071 tests, 1 error, 1 failure, 20 ignored (10 assumptions)

Total time: 171 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message