lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_20) - Build # 4461 - Still Failing!
Date Sun, 30 Nov 2014 10:07:37 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4461/
Java: 32bit/jdk1.8.0_20 -server -XX:+UseParallelGC (asserts: false)

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteInactiveReplicaTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteInactiveReplicaTest:     1) Thread[id=1884, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteInactiveReplicaTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteInactiveReplicaTest: 
   1) Thread[id=1884, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteInactiveReplicaTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest:     1) Thread[id=13675, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest: 
   1) Thread[id=13675, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)


FAILED:  org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)




Build Log:
[...truncated 8643 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteInactiveReplicaTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\init-core-data-001
   [junit4]   2> 662906 T1881 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 662906 T1881 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /p_gg/dv
   [junit4]   2> 662908 T1881 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 662908 T1881 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 662909 T1882 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 663010 T1881 oasc.ZkTestServer.run start zk server on port:50887
   [junit4]   2> 663010 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 663012 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 663016 T1889 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c4c909 name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663016 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 663016 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 663016 T1881 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 663023 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 663025 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 663028 T1892 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b55e8b name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663029 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 663029 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 663029 T1881 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 663032 T1881 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 663035 T1881 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 663037 T1881 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 663039 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 663039 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 663043 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 663043 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 663046 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663046 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663049 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 663049 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 663052 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 663052 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 663055 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 663055 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 663058 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 663058 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 663061 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663061 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663064 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663064 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663067 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 663067 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 663070 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 663070 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 663074 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 663076 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 663081 T1895 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1997f53 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663081 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 663081 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 664215 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 664223 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 664224 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50897
   [junit4]   2> 664228 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 664228 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 664229 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002
   [junit4]   2> 664229 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\'
   [junit4]   2> 664255 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\solr.xml
   [junit4]   2> 664266 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002
   [junit4]   2> 664267 T1881 oasc.CoreContainer.<init> New CoreContainer 14486680
   [junit4]   2> 664267 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\]
   [junit4]   2> 664267 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 664268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 664268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 664270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 664270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 664270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 664274 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 664277 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 664277 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 664277 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 664278 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 664278 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 664278 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 664290 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 664294 T1907 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a185aa name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 664295 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 664295 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 664301 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 664304 T1910 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1855eb0 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 664304 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 664307 T1881 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 664312 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 664317 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 664322 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 664326 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 664333 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 664336 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50897_p_gg%2Fdv
   [junit4]   2> 664337 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50897_p_gg%2Fdv
   [junit4]   2> 664341 T1881 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 664345 T1881 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 664348 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 664351 T1881 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:50897_p_gg%2Fdv
   [junit4]   2> 664352 T1881 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 664355 T1881 oasc.Overseer.start Overseer (id=92886067165528068-127.0.0.1:50897_p_gg%2Fdv-n_0000000000) starting
   [junit4]   2> 664359 T1881 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 664377 T1881 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 664380 T1881 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 664380 T1912 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 664384 T1881 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 664387 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 664393 T1911 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 664393 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002
   [junit4]   2> 664701 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\conf\
   [junit4]   2> 664756 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 664758 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 664758 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 664778 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 664781 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 664783 T1917 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11cda57 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 664783 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 664784 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 664787 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 664793 T1881 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 666145 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 666146 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 666155 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 666156 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50909
   [junit4]   2> 666159 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 666160 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 666160 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003
   [junit4]   2> 666160 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\'
   [junit4]   2> 666191 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\solr.xml
   [junit4]   2> 666201 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003
   [junit4]   2> 666202 T1881 oasc.CoreContainer.<init> New CoreContainer 11913574
   [junit4]   2> 666202 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\]
   [junit4]   2> 666203 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 666203 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 666203 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 666204 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 666204 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 666204 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 666205 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 666205 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 666205 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 666206 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 666209 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 666211 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 666211 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 666211 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 666212 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 666212 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 666212 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 666213 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 666229 T1929 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@115393e name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 666230 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 666230 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 666232 T1883 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x149ff62cdfc0006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 666234 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 666240 T1932 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fa8465 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 666240 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 666250 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 667259 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50909_p_gg%2Fdv
   [junit4]   2> 667261 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50909_p_gg%2Fdv
   [junit4]   2> 667272 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 667277 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003
   [junit4]   2> 667960 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\conf\
   [junit4]   2> 668075 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 668077 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 668077 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 669281 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 669281 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 669286 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 669288 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50918
   [junit4]   2> 669292 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 669293 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 669293 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004
   [junit4]   2> 669293 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\'
   [junit4]   2> 669324 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\solr.xml
   [junit4]   2> 669335 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004
   [junit4]   2> 669336 T1881 oasc.CoreContainer.<init> New CoreContainer 19289259
   [junit4]   2> 669336 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\]
   [junit4]   2> 669337 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 669337 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 669337 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 669339 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 669339 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 669339 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 669343 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 669346 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 669347 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 669347 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 669347 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 669348 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 669348 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 669351 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 669353 T1945 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ee49f name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 669354 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 669354 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 669358 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 669359 T1883 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x149ff62cdfc0008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 669362 T1948 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67030b name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 669362 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 669374 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 670381 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50918_p_gg%2Fdv
   [junit4]   2> 670383 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50918_p_gg%2Fdv
   [junit4]   2> 670393 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 670398 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004
   [junit4]   2> 670949 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\conf\
   [junit4]   2> 671009 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 671011 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 671011 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 672200 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 672201 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 672207 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 672209 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50927
   [junit4]   2> 672214 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 672214 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 672215 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005
   [junit4]   2> 672215 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\'
   [junit4]   2> 672252 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\solr.xml
   [junit4]   2> 672266 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005
   [junit4]   2> 672267 T1881 oasc.CoreContainer.<init> New CoreContainer 7043761
   [junit4]   2> 672267 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\]
   [junit4]   2> 672268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 672268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 672268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 672269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 672269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 672269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 672270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 672270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 672270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 672271 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 672280 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 672282 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 672283 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 672283 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 672284 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 672284 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 672284 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 672286 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672290 T1961 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c4587 name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672291 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672291 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 672296 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672300 T1964 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dd9180 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672301 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672314 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 673323 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50927_p_gg%2Fdv
   [junit4]   2> 673324 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50927_p_gg%2Fdv
   [junit4]   2> 673339 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 673344 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005
   [junit4]   2> 673974 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\conf\
   [junit4]   2> 674141 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 674142 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 674143 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 675358 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 675358 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 675366 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 675368 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50936
   [junit4]   2> 675373 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 675373 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 675373 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006
   [junit4]   2> 675373 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\'
   [junit4]   2> 675398 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\solr.xml
   [junit4]   2> 675412 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006
   [junit4]   2> 675413 T1881 oasc.CoreContainer.<init> New CoreContainer 31822724
   [junit4]   2> 675413 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\]
   [junit4]   2> 675413 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 675415 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 675415 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 675417 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 675419 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 675419 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 675419 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 675419 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 675420 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 675420 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 675421 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 675425 T1977 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2092d5 name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675426 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 675426 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 675431 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 675433 T1980 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@110b7ca name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675434 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 675444 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 676449 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50936_p_gg%2Fdv
   [junit4]   2> 676452 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50936_p_gg%2Fdv
   [junit4]   2> 676464 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 676472 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006
   [junit4]   2> 677561 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\conf\
   [junit4]   2> 677778 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 677780 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 677781 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 677787 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 677788 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 677791 T1984 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@87324c name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 677792 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 677792 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 677794 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 677852 T1902 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=clusterprop&wt=javabin&version=2} status=0 QTime=6 
   [junit4]   2> 677853 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 677859 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 0, message = {
   [junit4]   2> 	  "operation":"clusterprop",
   [junit4]   2> 	  "name":"legacyCloud",
   [junit4]   2> 	  "val":"false"}
   [junit4]   2> 677864 T1911 oasco.CollectionMutator.checkKeyExistence ERROR Skipping invalid Overseer message because it has no collection specified: {
   [junit4]   2> 	  "operation":"clusterprop",
   [junit4]   2> 	  "name":"legacyCloud",
   [junit4]   2> 	  "val":"false"}
   [junit4]   2> 677867 T1911 oasc.Overseer$ClusterStateUpdater.run ERROR Overseer could not process the current clusterstate state update message, skipping the message. org.apache.solr.common.SolrException: Overseer SETREPLICAPROPERTY requires collection and shard and replica and property and property.value no action taken.
   [junit4]   2> 	at org.apache.solr.cloud.overseer.ReplicaMutator.addReplicaProperty(ReplicaMutator.java:96)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:444)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:323)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 677870 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 677872 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 677876 T1987 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@129fc43 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 677876 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 677877 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 677879 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 677894 T1904 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&maxShardsPerNode=2&name=delDeadColl&action=CREATE&numShards=2&wt=javabin&version=2
   [junit4]   2> 677896 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected
   [junit4]   2> 677897 T1912 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delDeadColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 677898 T1988 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delDeadColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 677902 T1988 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 677902 T1988 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/delDeadColl 
   [junit4]   2> 677903 T1988 oascc.SolrZkClient.makePath makePath: /collections/delDeadColl
   [junit4]   2> 677908 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 677911 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delDeadColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 677911 T1911 oasco.ClusterStateMutator.createCollection building a new cName: delDeadColl
   [junit4]   2> 677917 T1948 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677918 T1910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677918 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677919 T1980 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677920 T1932 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677920 T1984 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677926 T1917 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677926 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678009 T1988 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection delDeadColl, shardNames [shard1, shard2] , replicationFactor : 2
   [junit4]   2> 678009 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard1_replica1 as part of slice shard1 of collection delDeadColl on 127.0.0.1:50909_p_gg%2Fdv
   [junit4]   2> 678013 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678014 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard1_replica2 as part of slice shard1 of collection delDeadColl on 127.0.0.1:50936_p_gg%2Fdv
   [junit4]   2> 678016 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv"}
   [junit4]   2> 678016 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv"} 
   [junit4]   2> 678021 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard2_replica1 as part of slice shard2 of collection delDeadColl on 127.0.0.1:50927_p_gg%2Fdv
   [junit4]   2> 678024 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv"}
   [junit4]   2> 678025 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv"} 
   [junit4]   2> 678027 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678027 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard2_replica2 as part of slice shard2 of collection delDeadColl on 127.0.0.1:50918_p_gg%2Fdv
   [junit4]   2> 678031 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv"}
   [junit4]   2> 678032 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv"} 
   [junit4]   2> 678033 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678035 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv"}
   [junit4]   2> 678036 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv"} 
   [junit4]   2> 678039 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678041 T1948 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678041 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678041 T1917 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678041 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1980 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1984 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1932 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678197 T1974 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&name=delDeadColl_shard1_replica2&action=CREATE&numShards=2&collection=delDeadColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 678226 T1957 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&name=delDeadColl_shard2_replica1&action=CREATE&numShards=2&collection=delDeadColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 678239 T1940 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=delDeadColl_shard2_replica2&action=CREATE&numShards=2&collection=delDeadColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 678266 T1923 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=conf1&name=delDeadColl_shard1_replica1&action=CREATE&numShards=2&collection=delDeadColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 678342 T1974 oasc.ZkController.publish publishing core=delDeadColl_shard1_replica2 state=down collection=delDeadColl
   [junit4]   2> 678343 T1974 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard1_replica2
   [junit4]   2> 678344 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678345 T1957 oasc.ZkController.publish publishing core=delDeadColl_shard2_replica1 state=down collection=delDeadColl
   [junit4]   2> 678346 T1957 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard2_replica1
   [junit4]   2> 678347 T1940 oasc.ZkController.publish publishing core=delDeadColl_shard2_replica2 state=down collection=delDeadColl
   [junit4]   2> 678347 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50936_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678349 T1940 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard2_replica2
   [junit4]   2> 678349 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50936_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678355 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "core_node_name":"core_node3",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50927_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678358 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "core_node_name":"core_node3",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50927_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678362 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678364 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50918_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678365 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50918_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678370 T1923 oasc.ZkController.publish publishing core=delDeadColl_shard1_replica1 state=down collection=delDeadColl
   [junit4]   2> 678371 T1923 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard1_replica1
   [junit4]   2> 678372 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678373 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50909_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678375 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50909_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678481 T1948 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1984 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678483 T1917 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1932 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1980 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 679344 T1974 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679345 T1974 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679345 T1974 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679346 T1957 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679347 T1957 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679348 T1957 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679348 T1974 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679349 T1974 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\'
   [junit4]   2> 679349 T1940 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679352 T1940 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679352 T1940 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679353 T1957 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679357 T1957 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\delDeadColl_shard2_replica1\'
   [junit4]   2> 679362 T1940 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679362 T1940 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\'
   [junit4]   2> 679373 T1923 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679379 T1923 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679379 T1923 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679383 T1923 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679383 T1923 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\delDeadColl_shard1_replica1\'
   [junit4]   2> 679391 T1974 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679410 T1940 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679414 T1974 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679419 T1923 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679419 T1940 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679425 T1957 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679436 T1974 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679439 T1974 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679447 T1940 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679449 T1940 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679450 T1923 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679451 T1957 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679452 T1974 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679455 T1974 oass.IndexSchema.readSchema [delDeadColl_shard1_replica2] Schema name=test
   [junit4]   2> 679470 T1940 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679474 T1940 oass.IndexSchema.readSchema [delDeadColl_shard2_replica2] Schema name=test
   [junit4]   2> 679493 T1923 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679499 T1923 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679503 T1957 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679507 T1957 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679526 T1974 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679538 T1974 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679540 T1974 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679543 T1923 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679547 T1923 oass.IndexSchema.readSchema [delDeadColl_shard1_replica1] Schema name=test
   [junit4]   2> 679573 T1940 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679576 T1940 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679577 T1940 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679580 T1957 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679592 T1957 oass.IndexSchema.readSchema [delDeadColl_shard2_replica1] Schema name=test
   [junit4]   2> 679661 T1923 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679663 T1957 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679663 T1923 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679664 T1923 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679666 T1957 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679667 T1957 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679695 T1974 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679700 T1974 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679706 T1974 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679709 T1940 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679710 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679710 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679710 T1974 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679712 T1923 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679715 T1940 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679716 T1957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679717 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679717 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679717 T1974 oasc.CoreContainer.create Creating SolrCore 'delDeadColl_shard1_replica2' using configuration from collection delDeadColl
   [junit4]   2> 679717 T1974 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 679718 T1974 oasc.SolrCore.<init> [delDeadColl_shard1_replica2] Opening new SolrCore at ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\
   [junit4]   2> 679718 T1974 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c96c8
   [junit4]   2> 679719 T1974 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\
   [junit4]   2> 679719 T1974 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\index/
   [junit4]   2> 679719 T1974 oasc.SolrCore.initIndex WARN [delDeadColl_shard1_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\index' doesn't exist. Creating new index...
   [junit4]   2> 679720 T1974 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\index
   [junit4]   2> 679720 T1974 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8867238024671883]
   [junit4]   2> 679722 T1974 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@19be8be lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b83ff1),segFN=segments_1,generation=1}
   [junit4]   2> 679722 T1974 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 679723 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 679724 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 679724 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 679724 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 679725 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 679725 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 679725 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 679726 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 679726 T1974 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 679726 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679727 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679727 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679727 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679728 T1974 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 679728 T1974 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 679729 T1974 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 679729 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 679730 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 679730 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 679731 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 679731 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 679731 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 679732 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 679732 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 679732 T1974 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 679733 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 679733 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 679738 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679739 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679740 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679740 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679744 T1974 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 679744 T1974 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 679745 T1974 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=4, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.45034392590936567]
   [junit4]   2> 679745 T1974 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@19be8be lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b83ff1),segFN=segments_1,generation=1}
   [junit4]   2> 679745 T1974 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 679746 T1974 oass.SolrIndexSearcher.<init> Opening Searcher@b40dcf[delDeadColl_shard1_replica2] main
   [junit4]   2> 679746 T1974 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679749 T1923 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679752 T1940 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679754 T1957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679756 T1974 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679756 T1974 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 679758 T1923 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679759 T1974 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 679759 T1974 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 679759 T1974 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 679760 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679760 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679760 T1940 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679762 T1957 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679763 T1974 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 679763 T1974 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 679765 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679765 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679765 T1923 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679766 T1974 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 679767 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679767 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679767 T1940 oasc.CoreContainer.create Creating SolrCore 'delDeadColl_shard2_replica2' using configuration from collection delDeadColl
   [junit4]   2> 679767 T1940 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 679768 T1940 oasc.SolrCore.<init> [delDeadColl_shard2_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\data\
   [junit4]   2> 679768 T1940 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c96c8
   [junit4]   2> 679768 T1940 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\data\
   [junit4]   2> 679768 T1940 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\data\index/
   [junit4]   2> 679768 T1940 oasc.SolrCore.initIndex WARN [delDeadColl_shard2_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplica

[...truncated too long message...]

Schema, CurrencyFieldXmlFileTest, TestQueryUtils, OverseerStatusTest, SpellPossibilityIteratorTest, ResponseLogComponentTest, TestOmitPositions, DistributedFacetPivotSmallAdvancedTest, MergeStrategyTest, TestLMJelinekMercerSimilarityFactory, TestInitQParser, DateRangeFieldTest, TestSweetSpotSimilarityFactory, TestFileDictionaryLookup, RemoteQueryErrorTest, CustomCollectionTest, TestLRUCache, SpatialFilterTest, DistributedSpellCheckComponentTest, ChangedSchemaMergeTest, BJQParserTest, ChaosMonkeySafeLeaderTest, DateMathParserTest, TestRealTimeGet, AddSchemaFieldsUpdateProcessorFactoryTest, TestDistributedSearch, TestDistributedMissingSort, HdfsBasicDistributedZkTest, SuggesterTSTTest, SolrIndexConfigTest, TestMissingGroups, TestUpdate, TestBM25SimilarityFactory, TestSolrQueryParserDefaultOperatorResource, TestXIncludeConfig, UniqFieldsUpdateProcessorFactoryTest, TestAddFieldRealTimeGet, TestCodecSupport, TestSolrXmlPersistor, FileBasedSpellCheckerTest, HdfsUnloadDistributedZkTest, TestFastOutputStream, TestCustomSort, TestSolr4Spatial2, TestAnalyzeInfixSuggestions, TestExpandComponent, TestSchemaManager, TestQuerySenderListener, AnalysisErrorHandlingTest, CloudExitableDirectoryReaderTest, TermVectorComponentTest, TestStressVersions, SliceStateTest, SolrCoreTest, MBeansHandlerTest, TestPivotHelperCode, LeaderFailoverAfterPartitionTest, HighlighterMaxOffsetTest, TestClassNameShortening, ResourceLoaderTest, TestSolrJ, TestSolrConfigHandler, PeerSyncTest, TestLuceneMatchVersion, TestWriterPerf, OpenExchangeRatesOrgProviderTest, TestIndexingPerformance, TestManagedStopFilterFactory, TestStressRecovery, TestDynamicFieldCollectionResource, PingRequestHandlerTest, TestStandardQParsers, AnalysisAfterCoreReloadTest, TestModifyConfFiles, TestSolrCoreProperties, TestQueryTypes, TriLevelCompositeIdRoutingTest, ConvertedLegacyTest, DistributedFacetPivotLargeTest, ScriptEngineTest, CoreAdminHandlerTest, DistributedIntervalFacetingTest, TestSystemIdResolver, TestTrie, PrimUtilsTest, DisMaxRequestHandlerTest, TestPerFieldSimilarity, AsyncMigrateRouteKeyTest, TestCollectionAPI, TestSolrQueryParser, ExternalCollectionsTest, ClusterStateTest, SampleTest, HdfsDirectoryTest, TestCopyFieldCollectionResource, SyncSliceTest, ZkCLITest, SuggesterTest, ExitableDirectoryReaderTest, BasicDistributedZk2Test, TestHighlightDedupGrouping, TestSortingResponseWriter, PathHierarchyTokenizerFactoryTest, ChaosMonkeyNothingIsSafeTest, DocValuesTest, TestAtomicUpdateErrorCases, FacetPivotSmallTest, TestQuerySenderNoQuery, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, TestConfigOverlay, TestConfigSets, AssignTest, SOLR749Test, ZkSolrClientTest, TestManagedSchemaFieldResource, TestWordDelimiterFilterFactory, StatsComponentTest, SimpleMLTQParserTest, EchoParamsTest, TestFiltering, TestFieldTypeCollectionResource, OverseerCollectionProcessorTest, TestComponentsName, TestMergePolicyConfig, UpdateParamsTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestNRTOpen, HighlighterTest, TestLMDirichletSimilarityFactory, TestInfoStreamLogging, SignatureUpdateProcessorFactoryTest, TestZkChroot, DocumentBuilderTest, TestBulkSchemaConcurrent, HdfsWriteToMultipleCollectionsTest, TestShortCircuitedRequests, PluginInfoTest, HighlighterConfigTest, TestReplicaProperties, TestManagedResource, OutputWriterTest, UpdateRequestProcessorFactoryTest, TestSchemaVersionResource, SimpleFacetsTest, FieldAnalysisRequestHandlerTest, TestDocBasedVersionConstraints, TestCSVResponseWriter, SearchHandlerTest, NoCacheHeaderTest, TestNonNRTOpen, TestFaceting, JsonLoaderTest, SuggesterWFSTTest, ClusterStateUpdateTest, TestRequestStatusCollectionAPI, TestConfig, TestCoreDiscovery, TestBlendedInfixSuggestions, LoggingHandlerTest, TestFieldTypeResource, TestTolerantSearch, DistributedQueryElevationComponentTest, DirectSolrSpellCheckerTest, SSLMigrationTest, SpellCheckComponentTest, RegexBoostProcessorTest, CurrencyFieldOpenExchangeTest, ZkNodePropsTest, TestStressLucene, AddBlockUpdateTest, TestComplexPhraseQParserPlugin, HdfsRecoveryZkTest, TestRandomMergePolicy, TestUtils, TimeZoneUtilsTest, HdfsLockFactoryTest, SolrPluginUtilsTest, CoreAdminCreateDiscoverTest, SharedFSAutoReplicaFailoverUtilsTest, SystemInfoHandlerTest, TestSolrQueryParserResource, BasicFunctionalityTest, TestRestManager, BasicDistributedZkTest, UnloadDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, TestReplicationHandler, TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, AutoCommitTest, TestDistributedGrouping, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestMultiCoreConfBootstrap, TestReload, DistributedTermsComponentTest, TestRangeQuery, TestSolr4Spatial, TestGroupingSearch, SolrCmdDistributorTest, QueryElevationComponentTest, TestSort, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, TestIndexSearcher, SoftAutoCommitTest, ShowFileRequestHandlerTest, SolrIndexSplitterTest, SimplePostToolTest, SpellCheckCollatorTest, SuggesterFSTTest, SolrRequestParserTest, TestFoldingMultitermQuery, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, TestValueSourceCache, MoreLikeThisHandlerTest, RequiredFieldsTest, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, IndexSchemaTest, TestJmxIntegration, TestCollationField, ReturnFieldsTest, JSONWriterTest, QueryParsingTest, TestAnalyzedSuggestions, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, TestBinaryResponseWriter, TestLFUCache, AlternateDirectoryTest, TestSolrIndexConfig, CopyFieldTest, BadComponentTest, TestDocSet, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestPhraseSuggestions, TestCharFilters, SynonymTokenizerTest, TestDFRSimilarityFactory, TestIBSimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TestJmxMonitoredMap, URLClassifyProcessorTest, TestFastLRUCache, PreAnalyzedFieldTest, TestSuggestSpellingConverter, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, TestDocumentBuilder, UUIDFieldTest, FileUtilsTest, CircularListTest, CursorPagingTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, AliasIntegrationTest, CollectionsAPIAsyncDistributedZkTest, DeleteReplicaTest, DeleteShardTest, DistribCursorPagingTest, DistribDocExpirationUpdateProcessorTest, DistributedQueueTest, HttpPartitionTest, LeaderInitiatedRecoveryOnCommitTest, MigrateRouteKeyTest, MultiThreadedOCPTest, OverriddenZkACLAndCredentialsProvidersTest, ReplicationFactorTest, RollingRestartTest, SharedFSAutoReplicaFailoverTest, SimpleCollectionCreateDeleteTest, SolrXmlInZkTest, TestCloudInspectUtil, TestCloudPivotFacet, TestDistribDocBasedVersion, TestLeaderElectionZkExpiry, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest, HdfsCollectionsAPIDistributedZkTest, HdfsSyncSliceTest, StressHdfsTest, TestClusterStateMutator, CachingDirectoryFactoryTest, QueryResultKeyTest, TestInitParams, TestReloadAndDeleteDocs, TestShardHandlerFactory, TestSolrDynamicMBean, TestSolrXml, TestSolrXmlPersistence, TestReplicationHandlerBackup]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandlerBackup -Dtests.seed=4F62D3D25588F392 -Dtests.slow=true -Dtests.locale=es_MX -Dtests.timezone=HST -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s | TestReplicationHandlerBackup (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)
   [junit4] Completed in 7221.83s, 1 test, 2 errors <<< FAILURES!

[...truncated 195 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:519: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:467: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:510: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 451 suites, 1840 tests, 3 suite-level errors, 1 error, 56 ignored (31 assumptions)

Total time: 257 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_20 -server -XX:+UseParallelGC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Mime
View raw message