lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 638 - Still Failing
Date Thu, 25 Sep 2014 07:56:31 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/638/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:583)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12191 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/init-core-data-001
   [junit4]   2> 852085 T19477 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 852086 T19477 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 852090 T19477 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 852091 T19477 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 852091 T19478 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 852192 T19477 oasc.ZkTestServer.run start zk server on port:39649
   [junit4]   2> 852193 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 852194 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 852197 T19484 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2bf4b016 name:ZooKeeperConnection Watcher:127.0.0.1:39649 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 852198 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 852198 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 852198 T19477 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 852201 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 852203 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 852204 T19486 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61f3e788 name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 852204 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 852205 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 852205 T19477 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 852207 T19477 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 852209 T19477 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 852210 T19477 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 852212 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 852213 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 852216 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 852216 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 852320 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 852321 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 852323 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 852323 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 852325 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 852326 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 852328 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 852329 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 852331 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 852332 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 852333 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 852334 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 852336 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 852337 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 852338 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 852339 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 852341 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 852342 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 852344 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 852345 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 852346 T19488 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@524aa681 name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 852346 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 852347 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> Using old style solr.xml
   [junit4]   2> 852348 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 852349 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 852350 T19490 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@452e24d7 name:ZooKeeperConnection Watcher:127.0.0.1:39649 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 852350 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 852350 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 852351 T19477 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 852352 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 852353 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 852354 T19492 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@205de707 name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 852354 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 852354 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 852356 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 852357 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 852359 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 852360 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
   [junit4]   2> 852463 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 852463 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 852465 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 852466 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 852467 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 852468 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
   [junit4]   2> 852470 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 852470 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
   [junit4]   2> 852472 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 852473 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 852474 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 852475 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 852477 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 852477 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 852479 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 852480 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 852481 T19477 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 852482 T19477 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
   [junit4]   2> 854183 T19477 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 854190 T19477 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 854193 T19477 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:39655
   [junit4]   2> 854197 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 854198 T19477 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 854198 T19477 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-002
   [junit4]   2> 854199 T19477 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-002/'
   [junit4]   2> 854254 T19477 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-002/solr.xml
   [junit4]   2> 854365 T19477 oasc.CoreContainer.<init> New CoreContainer 1749421491
   [junit4]   2> 854366 T19477 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-002/]
   [junit4]   2> 854368 T19477 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 854368 T19477 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 854369 T19477 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 854369 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 854370 T19477 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 854370 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 854371 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 854372 T19477 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 854372 T19477 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 854376 T19477 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 854380 T19477 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 854380 T19477 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 854381 T19477 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 854382 T19477 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39649/solr
   [junit4]   2> 854382 T19477 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 854383 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 854384 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 854386 T19503 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a2ea17c name:ZooKeeperConnection Watcher:127.0.0.1:39649 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 854387 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 854388 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 854390 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 854392 T19505 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12af17af name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 854392 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 854394 T19477 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 854397 T19477 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 854400 T19477 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 854403 T19477 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 854406 T19477 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 854410 T19477 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 854411 T19477 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39655_
   [junit4]   2> 854412 T19477 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39655_
   [junit4]   2> 854415 T19477 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 854417 T19477 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 854418 T19477 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 854420 T19477 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:39655_
   [junit4]   2> 854421 T19477 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 854423 T19477 oasc.Overseer.start Overseer (id=92512414122835974-127.0.0.1:39655_-n_0000000000) starting
   [junit4]   2> 854426 T19477 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 854435 T19477 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 854436 T19507 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 854437 T19477 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 854440 T19477 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 854442 T19477 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 854446 T19506 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 854451 T19477 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3
   [junit4]   2> 854451 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 854457 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 854458 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 854460 T19511 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2307dd5c name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 854461 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 854461 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 854463 T19477 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 854466 T19477 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 854466 T19477 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
   [junit4]   2> 854467 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 854468 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 854470 T19513 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2771407a name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 854470 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 854470 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 854473 T19514 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 854477 T19506 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1
   [junit4]   2> 854477 T19506 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 854477 T19506 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2
   [junit4]   2> 854481 T19506 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json
   [junit4]   2> 854483 T19514 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 854483 T19515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 856213 T19477 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 856214 T19477 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 856219 T19477 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 856221 T19477 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:44310
   [junit4]   2> 856225 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 856226 T19477 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 856226 T19477 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003
   [junit4]   2> 856226 T19477 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/'
   [junit4]   2> 856299 T19477 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/solr.xml
   [junit4]   2> 856399 T19477 oasc.CoreContainer.<init> New CoreContainer 893886510
   [junit4]   2> 856400 T19477 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/]
   [junit4]   2> 856401 T19477 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 856402 T19477 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 856402 T19477 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 856403 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 856403 T19477 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 856403 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 856404 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 856404 T19477 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 856404 T19477 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 856407 T19477 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 856410 T19477 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 856411 T19477 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 856411 T19477 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 856411 T19477 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39649/solr
   [junit4]   2> 856412 T19477 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 856412 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 856413 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 856415 T19526 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f10f842 name:ZooKeeperConnection Watcher:127.0.0.1:39649 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 856416 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 856417 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 856419 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 856421 T19528 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c824f93 name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 856421 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 856427 T19477 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 857436 T19477 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44310_
   [junit4]   2> 857437 T19477 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44310_
   [junit4]   2> 857443 T19477 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 857449 T19477 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3
   [junit4]   2> 857449 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 859348 T19477 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 859349 T19477 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 859355 T19477 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 859358 T19477 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:23716
   [junit4]   2> 859362 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 859362 T19477 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 859362 T19477 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004
   [junit4]   2> 859363 T19477 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/'
   [junit4]   2> 859420 T19477 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/solr.xml
   [junit4]   2> 859510 T19477 oasc.CoreContainer.<init> New CoreContainer 1966064132
   [junit4]   2> 859511 T19477 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/]
   [junit4]   2> 859512 T19477 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 859513 T19477 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 859513 T19477 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 859514 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 859514 T19477 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 859514 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 859515 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 859515 T19477 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 859515 T19477 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 859519 T19477 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 859521 T19477 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 859522 T19477 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 859522 T19477 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 859522 T19477 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39649/solr
   [junit4]   2> 859523 T19477 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 859523 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 859524 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 859526 T19541 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17244564 name:ZooKeeperConnection Watcher:127.0.0.1:39649 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 859528 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 859529 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 859532 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 859533 T19543 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cdfdb32 name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 859533 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 859626 T19477 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 860633 T19477 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:23716_
   [junit4]   2> 860634 T19477 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:23716_
   [junit4]   2> 860641 T19477 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 860649 T19477 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3
   [junit4]   2> 860650 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 862608 T19477 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 862609 T19477 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 862616 T19477 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 862618 T19477 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:63735
   [junit4]   2> 862622 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 862623 T19477 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 862623 T19477 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005
   [junit4]   2> 862624 T19477 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/'
   [junit4]   2> 862677 T19477 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/solr.xml
   [junit4]   2> 862771 T19477 oasc.CoreContainer.<init> New CoreContainer 1000581878
   [junit4]   2> 862772 T19477 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/]
   [junit4]   2> 862774 T19477 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 862775 T19477 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 862775 T19477 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 862776 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 862776 T19477 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 862776 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 862777 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 862777 T19477 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 862778 T19477 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 862781 T19477 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 862783 T19477 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 862784 T19477 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 862784 T19477 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 862785 T19477 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39649/solr
   [junit4]   2> 862785 T19477 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 862785 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 862786 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 862788 T19556 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@792ec7e6 name:ZooKeeperConnection Watcher:127.0.0.1:39649 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 862790 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 862790 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 862793 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 862794 T19558 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2395350f name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 862795 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 862801 T19477 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 863807 T19477 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63735_
   [junit4]   2> 863808 T19477 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63735_
   [junit4]   2> 863816 T19477 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 863822 T19477 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3
   [junit4]   2> 863823 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 865439 T19477 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 865440 T19477 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 865446 T19477 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 865448 T19477 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:37396
   [junit4]   2> 865451 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 865452 T19477 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 865452 T19477 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006
   [junit4]   2> 865453 T19477 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/'
   [junit4]   2> 865505 T19477 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/solr.xml
   [junit4]   2> 865591 T19477 oasc.CoreContainer.<init> New CoreContainer 1210130804
   [junit4]   2> 865591 T19477 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/]
   [junit4]   2> 865593 T19477 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 865593 T19477 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 865593 T19477 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 865594 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 865594 T19477 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 865594 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 865595 T19477 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 865595 T19477 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 865595 T19477 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 865598 T19477 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 865601 T19477 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 865601 T19477 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 865601 T19477 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 865602 T19477 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39649/solr
   [junit4]   2> 865602 T19477 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 865603 T19477 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 865604 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 865605 T19571 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d5e86c5 name:ZooKeeperConnection Watcher:127.0.0.1:39649 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 865607 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 865607 T19477 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 865609 T19477 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 865611 T19573 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@744c254f name:ZooKeeperConnection Watcher:127.0.0.1:39649/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 865611 T19477 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 865617 T19477 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 866624 T19477 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37396_
   [junit4]   2> 866626 T19477 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37396_
   [junit4]   2> 866632 T19477 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 866639 T19477 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3
   [junit4]   2> 866639 T19477 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 866798 T19520 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&name=solrj_collection&collection.configName=conf1&router.field=myOwnField&action=CREATE&numShards=2&wt=javabin&version=2
   [junit4]   2> 866799 T19514 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 866801 T19507 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField"}
   [junit4]   2> 866803 T19577 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField"}
   [junit4]   2> 866804 T19577 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection 
   [junit4]   2> 866805 T19577 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
   [junit4]   2> 866808 T19514 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 866812 T19506 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
   [junit4]   2> 866812 T19506 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
   [junit4]   2> 866813 T19506 oasc.Overseer$ClusterStateUpdater.createCollection state version solrj_collection 1
   [junit4]   2> 866816 T19574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 866816 T19529 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 866816 T19544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 866816 T19514 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 866816 T19515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 866816 T19559 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 866909 T19577 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection solrj_collection, shardNames [shard1, shard2] , replicationFactor : 2
   [junit4]   2> 866910 T19577 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:63735_
   [junit4]   2> 866911 T19577 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:23716_
   [junit4]   2> 866913 T19577 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:44310_
   [junit4]   2> 866917 T19577 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:37396_
   [junit4]   2> 867046 T19548 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard1_replica1&collection.configName=conf1&action=CREATE&shard=shard1&numShards=2&collection=solrj_collection&wt=javabin&version=2&qt=/admin/cores
   [junit4]   2> 867049 T19534 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard1_replica2&collection.configName=conf1&action=CREATE&shard=shard1&numShards=2&collection=solrj_collection&wt=javabin&version=2&qt=/admin/cores
   [junit4]   2> 867049 T19518 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard2_replica1&collection.configName=conf1&action=CREATE&shard=shard2&numShards=2&collection=solrj_collection&wt=javabin&version=2&qt=/admin/cores
   [junit4]   2> 867050 T19548 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection
   [junit4]   2> 867050 T19563 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard2_replica2&collection.configName=conf1&action=CREATE&shard=shard2&numShards=2&collection=solrj_collection&wt=javabin&version=2&qt=/admin/cores
   [junit4]   2> 867051 T19548 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 867051 T19514 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 867053 T19534 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection
   [junit4]   2> 867054 T19534 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 867054 T19563 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection
   [junit4]   2> 867054 T19518 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection
   [junit4]   2> 867055 T19563 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 867056 T19506 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:63735",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "node_name":"127.0.0.1:63735_",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica1"}
   [junit4]   2> 867056 T19518 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 867059 T19506 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:23716",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "node_name":"127.0.0.1:23716_",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica2"}
   [junit4]   2> 867061 T19514 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 867062 T19506 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:37396",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "node_name":"127.0.0.1:37396_",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica2"}
   [junit4]   2> 867065 T19506 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:44310",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "node_name":"127.0.0.1:44310_",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica1"}
   [junit4]   2> 867067 T19514 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 867068 T19529 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 867068 T19514 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 867068 T19544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 867068 T19559 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 867068 T19574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 867068 T19515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 868053 T19548 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
   [junit4]   2> 868054 T19548 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 868055 T19548 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 868055 T19548 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 868055 T19534 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
   [junit4]   2> 868056 T19534 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 868056 T19548 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 868057 T19534 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 868057 T19548 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/solrj_collection_shard1_replica1/'
   [junit4]   2> 868057 T19563 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
   [junit4]   2> 868057 T19534 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 868058 T19563 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 868058 T19518 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
   [junit4]   2> 868059 T19563 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 868059 T19518 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 868059 T19534 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 868060 T19518 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 868060 T19534 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/solrj_collection_shard1_replica2/'
   [junit4]   2> 868059 T19563 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 868060 T19518 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 868062 T19563 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 868063 T19518 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 868064 T19563 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/solrj_collection_shard2_replica2/'
   [junit4]   2> 868064 T19518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/solrj_collection_shard2_replica1/'
   [junit4]   2> 868126 T19548 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 868133 T19534 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 868145 T19563 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 868147 T19518 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 868191 T19548 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 868197 T19534 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 868207 T19563 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 868222 T19518 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 868294 T19548 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 868299 T19534 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 868307 T19548 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
   [junit4]   2> 868310 T19563 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 868314 T19534 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
   [junit4]   2> 868325 T19518 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 868337 T19563 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
   [junit4]   2> 868343 T19518 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
   [junit4]   2> 868921 T19534 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 868938 T19534 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 868941 T19548 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 868943 T19534 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 868950 T19548 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 868951 T19518 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 868954 T19548 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 868958 T19534 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 868961 T19518 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 868965 T19518 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 868965 T19534 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 868972 T19534 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 868973 T19548 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 868974 T19534 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 868977 T19534 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 868978 T19534 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 868979 T19534 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 868979 T19534 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 868980 T19534 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using configuration from collection solrj_collection
   [junit4]   2> 868980 T19534 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 868981 T19534 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/solrj_collection_shard1_replica2/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/solrj_collection_shard1_replica2/data/
   [junit4]   2> 868981 T19534 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44a9674f
   [junit4]   2> 868982 T19563 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 868983 T19518 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 868983 T19534 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/solrj_collection_shard1_replica2/data
   [junit4]   2> 868983 T19548 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 868984 T19534 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/solrj_collection_shard1_replica2/data/index/
   [junit4]   2> 868985 T19534 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 868986 T19534 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-004/solrj_collection_shard1_replica2/data/index
   [junit4]   2> 868986 T19534 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=45.59765625, floorSegmentMB=0.658203125, forceMergeDeletesPctAllowed=22.468454961642934, segmentsPerTier=25.0, maxCFSSegmentSizeMB=0.5390625, noCFSRatio=0.8789252069963214
   [junit4]   2> 868988 T19534 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-57965966)),segFN=segments_1,generation=1}
   [junit4]   2> 868988 T19534 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 868989 T19518 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 868991 T19563 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 868991 T19548 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 868995 T19563 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 868996 T19534 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 868996 T19548 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 868997 T19534 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 868997 T19548 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 868998 T19534 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 868998 T19548 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 868999 T19534 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 869000 T19534 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 869000 T19548 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 869000 T19534 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 869000 T19548 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 869001 T19534 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 869001 T19548 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using configuration from collection solrj_collection
   [junit4]   2> 869002 T19534 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 869002 T19548 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 869002 T19518 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 869003 T19548 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/solrj_collection_shard1_replica1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/solrj_collection_shard1_replica1/data/
   [junit4]   2> 869003 T19534 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 869004 T19518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 869004 T19548 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44a9674f
   [junit4]   2> 869005 T19534 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 869005 T19518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 869006 T19534 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 869006 T19548 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/solrj_collection_shard1_replica1/data
   [junit4]   2> 869007 T19534 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 869006 T19518 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 869008 T19534 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 869007 T19548 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/solrj_collection_shard1_replica1/data/index/
   [junit4]   2> 869009 T19534 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 869009 T19548 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 869009 T19518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 869010 T19548 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-005/solrj_collection_shard1_replica1/data/index
   [junit4]   2> 869008 T19563 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 869011 T19548 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=45.59765625, floorSegmentMB=0.658203125, forceMergeDeletesPctAllowed=22.468454961642934, segmentsPerTier=25.0, maxCFSSegmentSizeMB=0.5390625, noCFSRatio=0.8789252069963214
   [junit4]   2> 869010 T19518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 869009 T19534 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 869012 T19518 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection
   [junit4]   2> 869013 T19534 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 869013 T19548 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-43445e28)),segFN=segments_1,generation=1}
   [junit4]   2> 869014 T19534 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 869013 T19518 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 869015 T19534 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 869015 T19518 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/solrj_collection_shard2_replica1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/solrj_collection_shard2_replica1/data/
   [junit4]   2> 869016 T19534 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 869014 T19548 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 869016 T19518 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44a9674f
   [junit4]   2> 869017 T19534 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 869018 T19518 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/solrj_collection_shard2_replica1/data
   [junit4]   2> 869018 T19563 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 869019 T19518 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/solrj_collection_shard2_replica1/data/index/
   [junit4]   2> 869020 T19518 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 869021 T19518 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-003/solrj_collection_shard2_replica1/data/index
   [junit4]   2> 869022 T19518 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=45.59765625, floorSegmentMB=0.658203125, forceMergeDeletesPctAllowed=22.468454961642934, segmentsPerTier=25.0, maxCFSSegmentSizeMB=0.5390625, noCFSRatio=0.8789252069963214
   [junit4]   2> 869023 T19548 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 869023 T19518 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-146fd545)),segFN=segments_1,generation=1}
   [junit4]   2> 869024 T19548 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 869024 T19518 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 869025 T19548 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 869026 T19548 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 869026 T19563 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 869026 T19548 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 869027 T19548 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 869027 T19563 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 869028 T19548 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 869028 T19563 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 869029 T19548 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 869029 T19563 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 869029 T19548 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 869030 T19563 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 869031 T19548 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 869031 T19563 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 869032 T19548 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 869032 T19563 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using configuration from collection solrj_collection
   [junit4]   2> 869032 T19518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 869033 T19548 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 869033 T19563 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 869033 T19518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 869034 T19563 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/solrj_collection_shard2_replica2/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/solrj_collection_shard2_replica2/data/
   [junit4]   2> 869034 T19548 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 869035 T19563 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44a9674f
   [junit4]   2> 869035 T19518 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 869036 T19548 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 869037 T19548 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 869037 T19518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 869038 T19548 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 869037 T19563 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/solrj_collection_shard2_replica2/data
   [junit4]   2> 869039 T19548 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 869038 T19518 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 869040 T19548 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 869040 T19518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 869039 T19563 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/solrj_collection_shard2_replica2/data/index/
   [junit4]   2> 869041 T19518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 869041 T19563 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 869041 T19548 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 869042 T19563 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001/tempDir-006/solrj_collection_shard2_replica2/data/index
   [junit4]   2> 869042 T19518 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 869043 T19563 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=45.59765625, floorSegmentMB=0.658203125, forceMergeDeletesPctAllowed=22.468454961642934, segmentsPerTier=25.0, maxCFSSegmentSizeMB=0.5390625, noCFSRatio=0.8789252069963214
   [junit4]   2> 869043 T19548 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 869043 T19534 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 869045 T19563 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		comm

[...truncated too long message...]

LE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not fully create collection: awholynewstresscollection_collection0_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BB68C1B970D86BDC]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1627)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1653)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:1037)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20042, name=Thread-3934, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not fully create collection: awholynewstresscollection_collection5_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BB68C1B970D86BDC]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1627)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1653)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:1037)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20040, name=Thread-3932, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not fully create collection: awholynewstresscollection_collection3_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BB68C1B970D86BDC]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1627)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1653)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:1037)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20038, name=Thread-3930, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not fully create collection: awholynewstresscollection_collection1_3
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BB68C1B970D86BDC]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1627)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1653)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:1037)Throwable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20039, name=Thread-3931, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not fully create collection: awholynewstresscollection_collection2_3
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BB68C1B970D86BDC]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1627)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1653)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:1037)
   [junit4]   2> 1759921 T19477 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-BB68C1B970D86BDC-001
   [junit4]   2> 907844 T19476 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {intDefault=Lucene41(blocksize=128), text=PostingsFormat(name=Direct), range_facet_l=OrdsLucene41(blocksize=128), _version_=Lucene41(blocksize=128), multiDefault=OrdsLucene41(blocksize=128), a_t=OrdsLucene41(blocksize=128), id=PostingsFormat(name=Memory doPackFST= true), timestamp=OrdsLucene41(blocksize=128), a_i1=OrdsLucene41(blocksize=128), other_tl1=Lucene41(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Asserting)}, sim=DefaultSimilarity, locale=da, timezone=Pacific/Midway
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=238994216,total=529006592
   [junit4]   2> NOTE: All tests run in this JVM: [TestExceedMaxTermLength, CoreContainerCoreInitFailuresTest, ReplicationFactorTest, TestFastLRUCache, TestLMJelinekMercerSimilarityFactory, FieldMutatingUpdateProcessorTest, TestInitParams, SolrIndexSplitterTest, AddBlockUpdateTest, SpellCheckComponentTest, TestNonNRTOpen, StressHdfsTest, TermVectorComponentDistributedTest, TestCursorMarkWithoutUniqueKey, SuggesterTSTTest, TestNonDefinedSimilarityFactory, AssignTest, ClusterStateUpdateTest, TestDistributedGrouping, TestImplicitCoreProperties, DocumentAnalysisRequestHandlerTest, TestCoreDiscovery, PluginInfoTest, CoreAdminHandlerTest, TestInfoStreamLogging, SharedFSAutoReplicaFailoverUtilsTest, ZkCLITest, HdfsUnloadDistributedZkTest, TestSolrXmlPersistence, HdfsWriteToMultipleCollectionsTest, CustomCollectionTest, TestMergePolicyConfig, SuggestComponentTest, WordBreakSolrSpellCheckerTest, TestRandomMergePolicy, TermVectorComponentTest, StandardRequestHandlerTest, DeleteShardTest, BadIndexSchemaTest, TestNoOpRegenerator, HdfsBasicDistributedZk2Test, TestShortCircuitedRequests, SolrXmlInZkTest, SearchHandlerTest, StatelessScriptUpdateProcessorFactoryTest, ModifyConfFileTest, TestFastWriter, TriLevelCompositeIdRoutingTest, RequestHandlersTest, TestLRUCache, DocExpirationUpdateProcessorFactoryTest, FieldAnalysisRequestHandlerTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed on J3 in 908.38s, 1 test, 1 error <<< FAILURES!

[...truncated 735 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:531: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:472: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1359: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:961: There were test failures: 430 suites, 1775 tests, 1 error, 54 ignored (10 assumptions)

Total time: 176 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-trunk #636
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 707557858 bytes
Compression is 0.0%
Took 2 min 0 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message