lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@sd-datasolutions.de>
Subject [JENKINS] Lucene-Solr-trunk-Linux (64bit/jrockit-jdk1.6.0_33-R28.2.4-4.1.0) - Build # 1982 - Failure!
Date Thu, 25 Oct 2012 09:19:46 GMT
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/1982/
Java: 64bit/jrockit-jdk1.6.0_33-R28.2.4-4.1.0 -XnoOpt

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionIntegrationTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([141DB14FC9DA2771]:0)


REGRESSION:  org.apache.solr.cloud.LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([141DB14FC9DA2771]:0)




Build Log:
[...truncated 8428 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.LeaderElectionIntegrationTest
[junit4:junit4]   2> 14 T105 oas.SolrTestCaseJ4.setUp ###Starting testLeaderElectionAfterClientTimeout
[junit4:junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577
[junit4:junit4]   2> 15 T105 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 16 T106 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 16 T106 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 17 T106 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 116 T105 oasc.ZkTestServer.run start zk server on port:55087
[junit4:junit4]   2> 117 T105 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:55087 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c479777
[junit4:junit4]   2> 117 T111 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:55087
[junit4:junit4]   2> 117 T105 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 118 T111 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:55087, initiating session
[junit4:junit4]   2> 118 T107 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:48103
[junit4:junit4]   2> 118 T107 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:48103
[junit4:junit4]   2> 118 T109 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 229 T109 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a96ca174b0000 with negotiated timeout 10000 for client /127.0.0.1:48103
[junit4:junit4]   2> 229 T111 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:55087, sessionid = 0x13a96ca174b0000, negotiated timeout = 10000
[junit4:junit4]   2> 230 T112 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c479777 name:ZooKeeperConnection Watcher:127.0.0.1:55087 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 231 T105 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 231 T105 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 235 T110 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13a96ca174b0000
[junit4:junit4]   2> 236 T112 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 236 T105 oaz.ZooKeeper.close Session: 0x13a96ca174b0000 closed
[junit4:junit4]   2> 236 T107 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:48103 which had sessionid 0x13a96ca174b0000
[junit4:junit4]   2> 236 T105 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:55087/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c44d2be
[junit4:junit4]   2> 237 T113 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:55087
[junit4:junit4]   2> 237 T105 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 238 T113 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:55087, initiating session
[junit4:junit4]   2> 238 T107 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:48104
[junit4:junit4]   2> 238 T107 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:48104
[junit4:junit4]   2> 240 T109 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a96ca174b0001 with negotiated timeout 10000 for client /127.0.0.1:48104
[junit4:junit4]   2> 240 T113 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:55087, sessionid = 0x13a96ca174b0001, negotiated timeout = 10000
[junit4:junit4]   2> 240 T114 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c44d2be name:ZooKeeperConnection Watcher:127.0.0.1:55087/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 241 T105 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 241 T105 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 244 T105 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 247 T105 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 249 T105 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 253 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 254 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 257 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 259 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 264 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 266 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 270 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 271 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 272 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 273 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 275 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 276 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 278 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 279 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 283 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 284 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 288 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 288 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 292 T105 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 293 T105 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 297 T110 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13a96ca174b0001
[junit4:junit4]   2> 298 T114 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 298 T107 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:48104 which had sessionid 0x13a96ca174b0001
[junit4:junit4]   2> 297 T105 oaz.ZooKeeper.close Session: 0x13a96ca174b0001 closed
[junit4:junit4]   2> 299 T105 oasc.LeaderElectionIntegrationTest.setUp ####SETUP_START testLeaderElectionAfterClientTimeout
[junit4:junit4]   2> 305 T105 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 306 T105 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 307 T105 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 308 T105 oasc.CoreContainer.<init> New CoreContainer 474505727
[junit4:junit4]   2> 323 T105 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 323 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 397 T105 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 421 T105 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:55087/solr
[junit4:junit4]   2> 422 T105 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:55087/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c122609
[junit4:junit4]   2> 423 T115 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:55087
[junit4:junit4]   2> 423 T105 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 423 T115 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:55087, initiating session
[junit4:junit4]   2> 423 T107 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:48107
[junit4:junit4]   2> 423 T107 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:48107
[junit4:junit4]   2> 424 T109 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a96ca174b0002 with negotiated timeout 3000 for client /127.0.0.1:48107
[junit4:junit4]   2> 424 T115 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:55087, sessionid = 0x13a96ca174b0002, negotiated timeout = 3000
[junit4:junit4]   2> 425 T116 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c122609 name:ZooKeeperConnection Watcher:127.0.0.1:55087/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 425 T105 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 426 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 427 T105 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 429 T105 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7000_solr
[junit4:junit4]   2> 429 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7000_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7000_solr
[junit4:junit4]   2> 429 T105 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7000_solr
[junit4:junit4]   2> 431 T105 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 431 T105 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 431 T105 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 431 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 432 T105 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 432 T105 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 432 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 432 T105 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 433 T105 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 433 T105 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 436 T105 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 438 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 439 T105 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 440 T105 oasc.Overseer.start Overseer (id=88548937405497346-127.0.0.1:7000_solr-n_0000000000) starting
[junit4:junit4]   2> 440 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 441 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 441 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 442 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 442 T118 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 443 T105 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 446 T105 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 446 T117 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 460 T105 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 463 T105 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 464 T105 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 464 T105 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 466 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 466 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 466 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 501 T105 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 556 T105 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 557 T105 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 563 T105 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1045 T105 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1055 T105 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1059 T105 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1067 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1070 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1073 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1074 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1076 T105 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7000/
[junit4:junit4]   2> 1077 T105 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c0a4b31
[junit4:junit4]   2> 1077 T105 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7000/index/
[junit4:junit4]   2> 1077 T105 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7000/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1078 T105 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7000/index forceNew:false
[junit4:junit4]   2> 1079 T105 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c1def2a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c1dfc05),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1080 T105 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 1080 T105 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1081 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1082 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1082 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1082 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1083 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1083 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1084 T105 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1084 T105 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1085 T105 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1085 T105 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1085 T105 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1086 T105 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1086 T105 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1086 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1087 T105 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1087 T105 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1087 T105 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1088 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1088 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1088 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1089 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1089 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1090 T105 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1090 T105 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1090 T105 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1092 T105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1094 T105 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1096 T105 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1098 T105 oass.SolrIndexSearcher.<init> Opening Searcher@1c157b41 main
[junit4:junit4]   2> 1098 T105 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 1098 T105 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1098 T105 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1099 T105 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1161 T105 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1186 T119 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c157b41 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1190 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1192 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:create cxid:0x53 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 1194 T105 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 1195 T105 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7000/solr shardId:shard1
[junit4:junit4]   2> 1200 T105 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 1204 T105 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 1207 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:delete cxid:0x62 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 1210 T105 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1210 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:create cxid:0x63 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1453 T117 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1459 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7000_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7000/solr"}
[junit4:junit4]   2> 1468 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:create cxid:0x6a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 1483 T116 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 1713 T105 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 1714 T105 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 1714 T105 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 1715 T105 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 1718 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0002 type:create cxid:0x88 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1986 T117 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1990 T116 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 2022 T105 oasc.ZkController.register We are http://127.0.0.1:7000/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 2024 T105 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 2024 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2025 T105 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2027 T105 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2027 T105 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2027 T105 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2027 T105 oasc.CoreContainer.<init> New CoreContainer 473109932
[junit4:junit4]   2> 2028 T105 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2028 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2042 T105 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2057 T105 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:55087/solr
[junit4:junit4]   2> 2057 T105 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:55087/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c0fe877
[junit4:junit4]   2> 2058 T120 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:55087
[junit4:junit4]   2> 2058 T105 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2058 T120 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:55087, initiating session
[junit4:junit4]   2> 2058 T107 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:48110
[junit4:junit4]   2> 2059 T107 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:48110
[junit4:junit4]   2> 2059 T109 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a96ca174b0003 with negotiated timeout 3000 for client /127.0.0.1:48110
[junit4:junit4]   2> 2059 T120 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:55087, sessionid = 0x13a96ca174b0003, negotiated timeout = 3000
[junit4:junit4]   2> 2060 T121 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c0fe877 name:ZooKeeperConnection Watcher:127.0.0.1:55087/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2060 T105 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2061 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2061 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2062 T105 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7001_solr
[junit4:junit4]   2> 2062 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7001_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7001_solr
[junit4:junit4]   2> 2063 T105 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7001_solr
[junit4:junit4]   2> 2065 T116 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 2065 T105 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2065 T105 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2065 T105 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2066 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2066 T105 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2066 T105 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2066 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2066 T105 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2067 T105 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2067 T105 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2072 T105 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2076 T105 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2076 T105 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 2077 T105 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 2077 T105 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 2078 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2078 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2079 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2112 T105 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 2166 T105 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2168 T105 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2173 T105 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2492 T117 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2493 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7000_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7000/solr"}
[junit4:junit4]   2> 2495 T116 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 2495 T121 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 2641 T105 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2648 T105 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2651 T105 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2662 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2665 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2668 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2669 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2670 T105 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7001/
[junit4:junit4]   2> 2670 T105 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c0a4b31
[junit4:junit4]   2> 2670 T105 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7001/index/
[junit4:junit4]   2> 2671 T105 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7001/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2671 T105 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7001/index forceNew:false
[junit4:junit4]   2> 2673 T105 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c725e21 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c726aec),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2673 T105 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2673 T105 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2675 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2675 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2676 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2676 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2676 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2677 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2677 T105 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2677 T105 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2678 T105 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2678 T105 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2679 T105 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2679 T105 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2680 T105 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2680 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2680 T105 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2681 T105 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2681 T105 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2682 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2682 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2683 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2683 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2684 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2684 T105 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2684 T105 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2685 T105 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2687 T105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2689 T105 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2690 T105 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 2692 T105 oass.SolrIndexSearcher.<init> Opening Searcher@1c676d90 main
[junit4:junit4]   2> 2692 T105 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 2692 T105 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2692 T105 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2693 T105 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2698 T105 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2743 T122 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c676d90 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2751 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2753 T105 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2753 T105 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7001/solr shardId:shard1
[junit4:junit4]   2> 2757 T105 oasc.ZkController.register We are http://127.0.0.1:7001/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 2758 T105 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 2758 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2760 T105 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2763 T105 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2763 T105 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2764 T105 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2764 T105 oasc.CoreContainer.<init> New CoreContainer 476255582
[junit4:junit4]   2> 2765 T105 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2765 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2803 T105 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2820 T105 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:55087/solr
[junit4:junit4]   2> 2820 T105 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:55087/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c15ae2c
[junit4:junit4]   2> 2821 T123 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:55087
[junit4:junit4]   2> 2821 T105 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2822 T123 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:55087, initiating session
[junit4:junit4]   2> 2822 T107 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:48111
[junit4:junit4]   2> 2823 T107 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:48111
[junit4:junit4]   2> 2824 T109 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a96ca174b0004 with negotiated timeout 3000 for client /127.0.0.1:48111
[junit4:junit4]   2> 2824 T123 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:55087, sessionid = 0x13a96ca174b0004, negotiated timeout = 3000
[junit4:junit4]   2> 2824 T124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c15ae2c name:ZooKeeperConnection Watcher:127.0.0.1:55087/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2825 T105 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2825 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2826 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2828 T105 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7002_solr
[junit4:junit4]   2> 2828 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7002_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7002_solr
[junit4:junit4]   2> 2829 T105 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7002_solr
[junit4:junit4]   2> 2831 T121 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2831 T116 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2832 T105 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2832 T105 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2833 T105 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2833 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2834 T105 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2834 T105 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2835 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2835 T105 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2836 T105 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2836 T105 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2847 T105 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2857 T105 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2858 T105 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 2859 T105 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 2859 T105 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 2861 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2862 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2863 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2904 T105 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 2955 T105 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2956 T105 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2961 T105 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3001 T117 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3002 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7001_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7001/solr"}
[junit4:junit4]   2> 3005 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7001_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7001/solr"}
[junit4:junit4]   2> 3007 T121 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 3007 T124 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 3007 T116 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 3428 T105 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3435 T105 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3439 T105 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3447 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3450 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3452 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3454 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3455 T105 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7002/
[junit4:junit4]   2> 3455 T105 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c0a4b31
[junit4:junit4]   2> 3456 T105 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7002/index/
[junit4:junit4]   2> 3456 T105 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7002/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3459 T105 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7002/index forceNew:false
[junit4:junit4]   2> 3461 T105 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.SimpleFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/index6312163122tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c3b69b7),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3461 T105 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3462 T105 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 3463 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 3463 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 3463 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 3463 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 3464 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 3464 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 3464 T105 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 3465 T105 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3465 T105 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3466 T105 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 3466 T105 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 3466 T105 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3467 T105 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 3467 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 3467 T105 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 3468 T105 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3468 T105 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3469 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3469 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3469 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3470 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3470 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3471 T105 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3471 T105 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 3471 T105 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 3473 T105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3475 T105 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 3476 T105 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 3479 T105 oass.SolrIndexSearcher.<init> Opening Searcher@1c2eb00f main
[junit4:junit4]   2> 3479 T105 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3480 T105 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3480 T105 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3480 T105 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 3485 T105 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 3519 T125 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c2eb00f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3522 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3524 T105 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3524 T105 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7002/solr shardId:shard1
[junit4:junit4]   2> 3526 T105 oasc.ZkController.register We are http://127.0.0.1:7002/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 3526 T105 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 3526 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3527 T105 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3529 T105 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3529 T105 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3529 T105 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 3529 T105 oasc.CoreContainer.<init> New CoreContainer 474892430
[junit4:junit4]   2> 3530 T105 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3530 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3545 T105 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3556 T105 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:55087/solr
[junit4:junit4]   2> 3556 T105 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:55087/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c56c0f8
[junit4:junit4]   2> 3557 T126 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:55087
[junit4:junit4]   2> 3557 T105 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3557 T126 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:55087, initiating session
[junit4:junit4]   2> 3557 T107 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:48113
[junit4:junit4]   2> 3558 T107 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:48113
[junit4:junit4]   2> 3558 T109 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a96ca174b0005 with negotiated timeout 3000 for client /127.0.0.1:48113
[junit4:junit4]   2> 3558 T126 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:55087, sessionid = 0x13a96ca174b0005, negotiated timeout = 3000
[junit4:junit4]   2> 3559 T127 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c56c0f8 name:ZooKeeperConnection Watcher:127.0.0.1:55087/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3559 T105 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3559 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0005 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3560 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0005 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3561 T105 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7003_solr
[junit4:junit4]   2> 3561 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0005 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7003_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7003_solr
[junit4:junit4]   2> 3562 T105 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7003_solr
[junit4:junit4]   2> 3563 T121 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 3563 T124 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 3563 T116 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 3563 T105 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3564 T105 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3564 T105 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3564 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3564 T105 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3565 T105 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3565 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3565 T105 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3565 T105 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3565 T105 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3572 T105 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3576 T105 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 3576 T105 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 3577 T105 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 3577 T105 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 3578 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 3578 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 3579 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3617 T105 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 3668 T105 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 3669 T105 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3674 T105 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4012 T117 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4012 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7002_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7002/solr"}
[junit4:junit4]   2> 4014 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7002_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7002/solr"}
[junit4:junit4]   2> 4017 T121 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 4017 T124 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 4017 T116 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 4017 T127 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 4139 T105 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4147 T105 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4150 T105 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4159 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4162 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4164 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4165 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4166 T105 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7003/
[junit4:junit4]   2> 4167 T105 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c0a4b31
[junit4:junit4]   2> 4167 T105 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7003/index/
[junit4:junit4]   2> 4168 T105 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7003/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4168 T105 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7003/index forceNew:false
[junit4:junit4]   2> 4170 T105 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@1c463bed lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c4649f5; maxCacheMB=0.1220703125 maxMergeSizeMB=0.4736328125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4170 T105 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4171 T105 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4172 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4172 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4173 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4173 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4173 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4174 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4174 T105 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4174 T105 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4175 T105 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4175 T105 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4176 T105 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4176 T105 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4177 T105 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4177 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4177 T105 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4178 T105 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4178 T105 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4178 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4179 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4179 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4180 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4180 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4181 T105 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4181 T105 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4182 T105 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 4183 T105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4185 T105 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 4186 T105 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 4188 T105 oass.SolrIndexSearcher.<init> Opening Searcher@1c48a5b6 main
[junit4:junit4]   2> 4188 T105 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 4189 T105 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4189 T105 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4190 T105 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 4261 T105 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4286 T128 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c48a5b6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4289 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4290 T105 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4290 T105 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7003/solr shardId:shard1
[junit4:junit4]   2> 4292 T105 oasc.ZkController.register We are http://127.0.0.1:7003/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 4292 T105 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 4293 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4294 T105 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4295 T105 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4295 T105 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4295 T105 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4296 T105 oasc.CoreContainer.<init> New CoreContainer 475494915
[junit4:junit4]   2> 4296 T105 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4296 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4316 T105 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4331 T105 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:55087/solr
[junit4:junit4]   2> 4331 T105 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:55087/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c11f815
[junit4:junit4]   2> 4332 T129 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:55087
[junit4:junit4]   2> 4332 T105 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4332 T129 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:55087, initiating session
[junit4:junit4]   2> 4332 T107 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:48116
[junit4:junit4]   2> 4333 T107 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:48116
[junit4:junit4]   2> 4333 T109 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a96ca174b0006 with negotiated timeout 3000 for client /127.0.0.1:48116
[junit4:junit4]   2> 4333 T129 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:55087, sessionid = 0x13a96ca174b0006, negotiated timeout = 3000
[junit4:junit4]   2> 4334 T130 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c11f815 name:ZooKeeperConnection Watcher:127.0.0.1:55087/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4334 T105 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4334 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0006 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4335 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0006 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4336 T105 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7004_solr
[junit4:junit4]   2> 4336 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0006 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7004_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7004_solr
[junit4:junit4]   2> 4337 T105 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7004_solr
[junit4:junit4]   2> 4338 T116 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 4338 T124 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 4338 T127 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 4338 T121 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 4339 T105 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4339 T105 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4339 T105 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4339 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4340 T105 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4340 T105 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4340 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4340 T105 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4341 T105 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4341 T105 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 4345 T105 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4350 T105 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 4350 T105 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4350 T105 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4350 T105 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4351 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 4352 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 4352 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4385 T105 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4436 T105 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4437 T105 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4442 T105 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4521 T117 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4521 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7003_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7003/solr"}
[junit4:junit4]   2> 4525 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7003_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7003/solr"}
[junit4:junit4]   2> 4527 T121 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 4527 T124 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 4527 T127 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 4527 T116 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 4528 T130 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 4905 T105 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4912 T105 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4915 T105 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4923 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4926 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4929 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4930 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4931 T105 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7004/
[junit4:junit4]   2> 4931 T105 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c0a4b31
[junit4:junit4]   2> 4932 T105 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7004/index/
[junit4:junit4]   2> 4932 T105 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7004/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4933 T105 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_7004/index forceNew:false
[junit4:junit4]   2> 4934 T105 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c3c5ee0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c3c6bab),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4934 T105 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4935 T105 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4936 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4937 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4937 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4937 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4938 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4938 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4938 T105 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4939 T105 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4939 T105 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4940 T105 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4940 T105 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4940 T105 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4941 T105 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4941 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4941 T105 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4942 T105 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4942 T105 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4943 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4943 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4943 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4944 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4944 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4945 T105 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4945 T105 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4946 T105 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 4948 T105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4949 T105 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 4950 T105 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 4952 T105 oass.SolrIndexSearcher.<init> Opening Searcher@1c37c114 main
[junit4:junit4]   2> 4952 T105 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 4953 T105 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4953 T105 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4954 T105 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 4958 T105 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4980 T131 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c37c114 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4983 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4985 T105 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4985 T105 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7004/solr shardId:shard1
[junit4:junit4]   2> 4987 T105 oasc.ZkController.register We are http://127.0.0.1:7004/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 4987 T105 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 4987 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4989 T105 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4990 T105 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4990 T105 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4991 T105 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4991 T105 oasc.CoreContainer.<init> New CoreContainer 476193588
[junit4:junit4]   2> 4992 T105 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4992 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5008 T105 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5019 T105 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:55087/solr
[junit4:junit4]   2> 5019 T105 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:55087/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c7104e2
[junit4:junit4]   2> 5020 T132 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:55087
[junit4:junit4]   2> 5020 T105 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5021 T132 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:55087, initiating session
[junit4:junit4]   2> 5021 T107 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:48117
[junit4:junit4]   2> 5021 T107 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:48117
[junit4:junit4]   2> 5022 T109 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a96ca174b0007 with negotiated timeout 3000 for client /127.0.0.1:48117
[junit4:junit4]   2> 5022 T132 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:55087, sessionid = 0x13a96ca174b0007, negotiated timeout = 3000
[junit4:junit4]   2> 5022 T133 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c7104e2 name:ZooKeeperConnection Watcher:127.0.0.1:55087/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5022 T105 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5023 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0007 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5023 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0007 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5024 T105 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:3333_solr
[junit4:junit4]   2> 5024 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0007 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:3333_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:3333_solr
[junit4:junit4]   2> 5025 T105 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:3333_solr
[junit4:junit4]   2> 5026 T121 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 5026 T116 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 5026 T127 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 5026 T124 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 5026 T130 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 5027 T105 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5027 T105 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5027 T105 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5028 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5028 T105 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5028 T105 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5028 T105 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5028 T105 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5029 T105 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5029 T105 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5030 T117 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5031 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7004_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7004/solr"}
[junit4:junit4]   2> 5032 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7004_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7004/solr"}
[junit4:junit4]   2> 5034 T105 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5035 T121 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 5035 T116 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 5035 T130 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 5035 T127 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 5035 T133 oaz.ClientCnxn$EventThread.processEvent SEVERE Error while calling watcher  java.lang.NullPointerException
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader$2.process(ZkStateReader.java:189)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4:junit4]   2> 
[junit4:junit4]   2> 5035 T124 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 5041 T105 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5041 T105 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5041 T105 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5042 T105 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5042 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5043 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 5043 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5079 T105 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 5131 T105 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5132 T105 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5137 T105 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5625 T105 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5632 T105 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5635 T105 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5647 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5650 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5653 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5654 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5655 T105 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_3333/
[junit4:junit4]   2> 5656 T105 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c0a4b31
[junit4:junit4]   2> 5656 T105 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_3333/index/
[junit4:junit4]   2> 5656 T105 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_3333/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5657 T105 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351149557577/data_3333/index forceNew:false
[junit4:junit4]   2> 5658 T105 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1c3fd8ae lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c3fe4c4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5659 T105 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5660 T105 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5664 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5664 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5664 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5665 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5665 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5665 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5666 T105 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5666 T105 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5666 T105 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5667 T105 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5667 T105 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5668 T105 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5668 T105 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5669 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5669 T105 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5670 T105 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5670 T105 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5670 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5671 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5671 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5672 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5672 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5673 T105 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5673 T105 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5674 T105 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5676 T105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5678 T105 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5679 T105 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5680 T105 oass.SolrIndexSearcher.<init> Opening Searcher@1c5c7393 main
[junit4:junit4]   2> 5681 T105 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 5681 T105 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5681 T105 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5682 T105 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5687 T105 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5710 T134 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c5c7393 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5713 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5714 T105 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 5714 T105 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:3333/solr shardId:shard2
[junit4:junit4]   2> 5715 T105 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 5718 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0007 type:delete cxid:0x40 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 5719 T105 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 5719 T110 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a96ca174b0007 type:create cxid:0x41 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6041 T117 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6042 T117 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:3333_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:3333/solr"}
[junit4:junit4]   2> 6049 T121 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 6049 T127 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 6049 T124 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 6049 T130 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 6049 T116 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 7200041 T104 ccr.ThreadLeakControl$2.evaluate WARNING Suite execution timed out: org.apache.solr.cloud.LeaderElectionIntegrationTest
[junit4:junit4]   2> 	==== jstack at approximately timeout time ====
[junit4:junit4]   2> 	"Main Thread" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@1c26eea4
[junit4:junit4]   2> 		at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 		- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@1c26eea4
[junit4:junit4]   2> 		at java.lang.Thread.join(Thread.java:1186)
[junit4:junit4]   2> 		at java.lang.Thread.join(Thread.java:1239)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:502)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:399)
[junit4:junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:161)
[junit4:junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:255)
[junit4:junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"(Signal Handler)" ID=2 RUNNABLE
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"(OC Main Thread)" ID=3 RUNNABLE
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"(Code Generation Thread 1)" ID=4 RUNNABLE
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"(Code Optimization Thread 1)" ID=5 RUNNABLE
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"(VM Periodic Task)" ID=6 RUNNABLE
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"Finalizer" ID=7 RUNNABLE
[junit4:junit4]   2> 		at jrockit.memory.Finalizer.waitForFinalizees(Native Method)
[junit4:junit4]   2> 		at jrockit.memory.Finalizer.access$700(Finalizer.java:12)
[junit4:junit4]   2> 		at jrockit.memory.Finalizer$4.run(Finalizer.java:189)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"Reference Handler" ID=8 RUNNABLE
[junit4:junit4]   2> 		at java.lang.ref.Reference.waitForActivatedQueue(Native Method)
[junit4:junit4]   2> 		at java.lang.ref.Reference.access$100(Reference.java:11)
[junit4:junit4]   2> 		at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:82)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"(Sensor Event Thread)" ID=9 RUNNABLE
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"VM JFR Buffer Thread" ID=10 RUNNABLE (in native code)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"JMAPI event thread" ID=14 RUNNABLE (in native code)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"JFR request timer" ID=15 WAITING on java.util.TaskQueue@1c072393
[junit4:junit4]   2> 		at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.TaskQueue@1c072393
[junit4:junit4]   2> 		at java.lang.Object.wait(Object.java:485)
[junit4:junit4]   2> 		at java.util.TimerThread.mainLoop(Timer.java:483)
[junit4:junit4]   2> 		at java.util.TimerThread.run(Timer.java:462)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TimeLimitedCollector timer thread" ID=62 TIMED_WAITING
[junit4:junit4]   2> 		at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 		at org.apache.lucene.search.TimeLimitingCollector$TimerThread.run(TimeLimitingCollector.java:267)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"SUITE-LeaderElectionIntegrationTest-seed#[141DB14FC9DA2771]" ID=104 RUNNABLE
[junit4:junit4]   2> 		at sun.management.ThreadImpl.dumpThreads0(Native Method)
[junit4:junit4]   2> 		at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:433)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:655)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$800(ThreadLeakControl.java:60)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:396)
[junit4:junit4]   2> 		- locked java.lang.Object@1c09d8e1
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:556)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:79)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:492)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]" ID=105 TIMED_WAITING
[junit4:junit4]   2> 		at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:311)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:144)
[junit4:junit4]   2> 		at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:151)
[junit4:junit4]   2> 		at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:95)
[junit4:junit4]   2> 		at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:261)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:733)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.register(ZkController.java:566)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.register(ZkController.java:532)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.registerInZk(CoreContainer.java:709)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.register(CoreContainer.java:693)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.load(CoreContainer.java:535)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
[junit4:junit4]   2> 		at org.apache.solr.cloud.LeaderElectionIntegrationTest.setupContainer(LeaderElectionIntegrationTest.java:152)
[junit4:junit4]   2> 		at org.apache.solr.cloud.LeaderElectionIntegrationTest.setUp(LeaderElectionIntegrationTest.java:107)
[junit4:junit4]   2> 		at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
[junit4:junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:771)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"Thread-26" ID=106 WAITING on org.apache.zookeeper.server.NIOServerCnxn$Factory@1c0c38b5
[junit4:junit4]   2> 		at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 		- waiting on org.apache.zookeeper.server.NIOServerCnxn$Factory@1c0c38b5
[junit4:junit4]   2> 		at java.lang.Thread.join(Thread.java:1186)
[junit4:junit4]   2> 		at java.lang.Thread.join(Thread.java:1239)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:103)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:214)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0" ID=107 RUNNABLE (in native code)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 		at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 		- locked sun.nio.ch.Util$2@1c0c3923
[junit4:junit4]   2> 		- locked java.util.Collections$UnmodifiableSet@1c0c3921
[junit4:junit4]   2> 		- locked sun.nio.ch.EPollSelectorImpl@1c0c38e1
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 		at org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:232)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"SessionTracker" ID=108 TIMED_WAITING on org.apache.zookeeper.server.SessionTrackerImpl@1c0c199f
[junit4:junit4]   2> 		at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 		- timed waiting on org.apache.zookeeper.server.SessionTrackerImpl@1c0c199f
[junit4:junit4]   2> 		at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:145)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"SyncThread:0" ID=109 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0c1a1a
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0c1a1a
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:94)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"ProcessThread:-1" ID=110 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0c1a5f
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0c1a5f
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:105)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087)" ID=115 RUNNABLE (in native code)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 		at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 		- locked sun.nio.ch.Util$2@1c0d91b0
[junit4:junit4]   2> 		- locked java.util.Collections$UnmodifiableSet@1c0d91ae
[junit4:junit4]   2> 		- locked sun.nio.ch.EPollSelectorImpl@1c0d919c
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread" ID=116 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0d9bcb
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0d9bcb
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"Thread-27" ID=117 TIMED_WAITING
[junit4:junit4]   2> 		at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 		at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:156)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"Overseer-88548937405497346-127.0.0.1:7000_solr-n_0000000000" ID=118 WAITING on java.util.concurrent.CountDownLatch$Sync@1c0daab9
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.CountDownLatch$Sync@1c0daab9
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
[junit4:junit4]   2> 		at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
[junit4:junit4]   2> 		at org.apache.solr.cloud.DistributedQueue$LatchChildWatcher.await(DistributedQueue.java:179)
[junit4:junit4]   2> 		at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:277)
[junit4:junit4]   2> 		at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:86)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"searcherExecutor-61-thread-1" ID=119 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0ad6d3
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0ad6d3
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087)" ID=120 RUNNABLE (in native code)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 		at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 		- locked sun.nio.ch.Util$2@1c0b3abc
[junit4:junit4]   2> 		- locked java.util.Collections$UnmodifiableSet@1c0b3aba
[junit4:junit4]   2> 		- locked sun.nio.ch.EPollSelectorImpl@1c0b3aa8
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread" ID=121 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0b2c64
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0b2c64
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"searcherExecutor-69-thread-1" ID=122 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0cca33
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0cca33
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087)" ID=123 RUNNABLE
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 		at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 		- locked sun.nio.ch.Util$2@1c0cac09
[junit4:junit4]   2> 		- locked java.util.Collections$UnmodifiableSet@1c0cac07
[junit4:junit4]   2> 		- locked sun.nio.ch.EPollSelectorImpl@1c0cabf5
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread" ID=124 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0ce5ab
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c0ce5ab
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"searcherExecutor-77-thread-1" ID=125 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c9b74ac
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c9b74ac
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087)" ID=126 RUNNABLE (in native code)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 		at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 		- locked sun.nio.ch.Util$2@1cb31ab2
[junit4:junit4]   2> 		- locked java.util.Collections$UnmodifiableSet@1cb31ab0
[junit4:junit4]   2> 		- locked sun.nio.ch.EPollSelectorImpl@1cb31a9e
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread" ID=127 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cb9281c
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cb9281c
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"searcherExecutor-85-thread-1" ID=128 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cb7ac8f
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cb7ac8f
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087)" ID=129 RUNNABLE (in native code)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 		at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 		- locked sun.nio.ch.Util$2@1cb7ec96
[junit4:junit4]   2> 		- locked java.util.Collections$UnmodifiableSet@1cb7ec94
[junit4:junit4]   2> 		- locked sun.nio.ch.EPollSelectorImpl@1cb7ec82
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread" ID=130 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cb7cca8
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cb7cca8
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"searcherExecutor-93-thread-1" ID=131 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cadbacc
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cadbacc
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087)" ID=132 RUNNABLE (in native code)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 		at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 		at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 		- locked sun.nio.ch.Util$2@1cc74b36
[junit4:junit4]   2> 		- locked java.util.Collections$UnmodifiableSet@1cc74b34
[junit4:junit4]   2> 		- locked sun.nio.ch.EPollSelectorImpl@1cc74b22
[junit4:junit4]   2> 		at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread" ID=133 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cc76b40
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cc76b40
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	"searcherExecutor-101-thread-1" ID=134 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c9b61b8
[junit4:junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1c9b61b8
[junit4:junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 	^^==============================================
[junit4:junit4]   2> 	
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderElectionIntegrationTest -Dtests.method=testLeaderElectionAfterClientTimeout -Dtests.seed=141DB14FC9DA2771 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=pl_PL -Dtests.timezone=Australia/Broken_Hill -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR   7200s J0 | LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([141DB14FC9DA2771]:0)
[junit4:junit4]   2> 7200117 T104 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 24 leaked thread(s).
[junit4:junit4]   2> 7220199 T104 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 24 threads leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionIntegrationTest: 
[junit4:junit4]   2> 	   1) Thread[id=125, name=searcherExecutor-77-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	   2) Thread[id=131, name=searcherExecutor-93-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	   3) Thread[id=132, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	   4) Thread[id=121, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	   5) Thread[id=119, name=searcherExecutor-61-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	   6) Thread[id=126, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	   7) Thread[id=133, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	   8) Thread[id=106, name=Thread-26, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.lang.Thread.join(Thread.java:1186)
[junit4:junit4]   2> 	        at java.lang.Thread.join(Thread.java:1239)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:103)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:214)
[junit4:junit4]   2> 	   9) Thread[id=116, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	  10) Thread[id=120, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	  11) Thread[id=107, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 	        at org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:232)
[junit4:junit4]   2> 	  12) Thread[id=129, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	  13) Thread[id=130, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	  14) Thread[id=122, name=searcherExecutor-69-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	  15) Thread[id=115, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	  16) Thread[id=108, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:145)
[junit4:junit4]   2> 	  17) Thread[id=110, name=ProcessThread:-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:105)
[junit4:junit4]   2> 	  18) Thread[id=105, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771], state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:311)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:144)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:151)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:95)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:261)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:733)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkController.register(ZkController.java:566)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkController.register(ZkController.java:532)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.registerInZk(CoreContainer.java:709)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.register(CoreContainer.java:693)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:535)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.LeaderElectionIntegrationTest.setupContainer(LeaderElectionIntegrationTest.java:152)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.LeaderElectionIntegrationTest.setUp(LeaderElectionIntegrationTest.java:107)
[junit4:junit4]   2> 	        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
[junit4:junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:771)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	  19) Thread[id=117, name=Thread-27, state=TIMED_WAITING, group=Overseer state updater.]
[junit4:junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:156)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	  20) Thread[id=124, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	  21) Thread[id=134, name=searcherExecutor-101-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	  22) Thread[id=128, name=searcherExecutor-85-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	  23) Thread[id=127, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 	  24) Thread[id=123, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
[junit4:junit4]   2> 	        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 7220206 T104 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=125, name=searcherExecutor-77-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	   2) Thread[id=131, name=searcherExecutor-93-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	   3) Thread[id=132, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	   4) Thread[id=121, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	   5) Thread[id=119, name=searcherExecutor-61-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	   6) Thread[id=126, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	   7) Thread[id=133, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	   8) Thread[id=106, name=Thread-26, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	   9) Thread[id=116, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  10) Thread[id=120, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  11) Thread[id=107, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  12) Thread[id=129, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  13) Thread[id=130, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  14) Thread[id=122, name=searcherExecutor-69-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  15) Thread[id=115, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  16) Thread[id=108, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  17) Thread[id=110, name=ProcessThread:-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  18) Thread[id=105, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771], state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  19) Thread[id=117, name=Thread-27, state=TIMED_WAITING, group=Overseer state updater.]
[junit4:junit4]   2> 	  20) Thread[id=124, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  21) Thread[id=134, name=searcherExecutor-101-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  22) Thread[id=128, name=searcherExecutor-85-thread-1, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  23) Thread[id=127, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	  24) Thread[id=123, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[141DB14FC9DA2771]-SendThread(localhost.localdomain:55087), state=RUNNABLE, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 7220207 T121 oaz.ClientCnxn$EventThread.run SEVERE Event thread exiting due to interruption java.lang.InterruptedException
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996)
[junit4:junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7220207 T116 oaz.ClientCnxn$EventThread.run SEVERE Event thread exiting due to interruption java.lang.InterruptedException
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996)
[junit4:junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7220208 T133 oaz.ClientCnxn$EventThread.run SEVERE Event thread exiting due to interruption java.lang.InterruptedException
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996)
[junit4:junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7220208 T121 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7220208 T116 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7220208 T133 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7220209 T108 oazs.SessionTrackerImpl.run SEVERE Unexpected interruption java.lang.InterruptedException
[junit4:junit4]   2> 	at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:145)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7220209 T108 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4]   2> 7220210 T130 oaz.ClientCnxn$EventThread.run SEVERE Event thread exiting due to interruption java.lang.InterruptedException
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996)
[junit4:junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7220210 T130 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7220210 T117 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=88548937405497346-127.0.0.1:7000_solr-n_0000000000) am no longer a leader.
[junit4:junit4]   2> 7220211 T110 oazs.PrepRequestProcessor.run SEVERE Unexpected interruption java.lang.InterruptedException
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996)
[junit4:junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:105)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7220211 T110 oazs.PrepRequestProcessor.run PrepRequestProcessor exited loop!
[junit4:junit4]   2> 7220211 T115 oaz.ClientCnxn$SendThread.run WARNING Session 0x13a96ca174b0002 for server localhost.localdomain/127.0.0.1:55087, unexpected error, closing socket connection and attempting reconnect java.nio.channels.ClosedByInterruptException
[junit4:junit4]   2> 	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:343)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:890)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1154)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7220215 T124 oaz.ClientCnxn$EventThread.run SEVERE Event thread exiting due to interruption java.lang.InterruptedException
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996)
[junit4:junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7220215 T107 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x13a96ca174b0002 due to java.nio.channels.ClosedByInterruptException
[junit4:junit4]   2> 7220215 T124 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7220216 T107 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:48107 which had sessionid 0x13a96ca174b0002
[junit4:junit4]   2> 7220216 T127 oaz.ClientCnxn$EventThread.run SEVERE Event thread exiting due to interruption java.lang.InterruptedException
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
[junit4:junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996)
[junit4:junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7220216 T127 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7220224 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7220224 T105 oasc.CoreContainer.registerInZk SEVERE  org.apache.solr.common.cloud.ZooKeeperException: 
[junit4:junit4]   2> 	at org.apache.solr.cloud.ZkController.register(ZkController.java:570)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ZkController.register(ZkController.java:532)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer.registerInZk(CoreContainer.java:709)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer.register(CoreContainer.java:694)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:535)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:357)
[junit4:junit4]   2> 	at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElectionIntegrationTest.setupContainer(LeaderElectionIntegrationTest.java:152)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElectionIntegrationTest.setUp(LeaderElectionIntegrationTest.java:107)
[junit4:junit4]   2> 	at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
[junit4:junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:771)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:683)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4:junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:44)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56)
[junit4:junit4]   2> 	at co

[...truncated too long message...]

perty(System.java:650)
[junit4:junit4]   2> 	        at com.sun.org.apache.xalan.internal.utils.SecuritySupport$4.run(SecuritySupport.java:135)
[junit4:junit4]   2> 	        at com.sun.org.apache.xalan.internal.utils.SecuritySupport.getSystemProperty(SecuritySupport.java:132)
[junit4:junit4]   2> 	        at com.sun.org.apache.xalan.internal.utils.ObjectFactory.lookUpFactoryClassName(ObjectFactory.java:304)
[junit4:junit4]   2> 	        at com.sun.org.apache.xalan.internal.utils.ObjectFactory.lookUpFactoryClass(ObjectFactory.java:249)
[junit4:junit4]   2> 	        at com.sun.org.apache.xalan.internal.utils.ObjectFactory.createObject(ObjectFactory.java:174)
[junit4:junit4]   2> 	        at com.sun.org.apache.xalan.internal.utils.ObjectFactory.createObject(ObjectFactory.java:144)
[junit4:junit4]   2> 	        at com.sun.org.apache.xml.internal.dtm.DTMManager.newInstance(DTMManager.java:144)
[junit4:junit4]   2> 	        at com.sun.org.apache.xpath.internal.XPathContext.init(XPathContext.java:337)
[junit4:junit4]   2> 	        at com.sun.org.apache.xpath.internal.XPathContext.<init>(XPathContext.java:313)
[junit4:junit4]   2> 	        at com.sun.org.apache.xpath.internal.XPathContext.<init>(XPathContext.java:309)
[junit4:junit4]   2> 	        at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.eval(XPathImpl.java:192)
[junit4:junit4]   2> 	        at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.evaluate(XPathImpl.java:266)
[junit4:junit4]   2> 	        at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:85)
[junit4:junit4]   2> 	        at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:43)
[junit4:junit4]   2> 	        at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:151)
[junit4:junit4]   2> 	        at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:371)
[junit4:junit4]   2> 	        at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:115)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.getSchemaFromZk(CoreContainer.java:1443)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:833)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:534)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.LeaderElectionIntegrationTest.setupContainer(LeaderElectionIntegrationTest.java:152)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.LeaderElectionIntegrationTest.setUp(LeaderElectionIntegrationTest.java:100)
[junit4:junit4]   2> 	        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
[junit4:junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:771)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=pl_PL, timezone=Australia/Broken_Hill
[junit4:junit4]   2> NOTE: Linux 3.2.0-32-generic amd64/Oracle Corporation 1.6.0_33 (64-bit)/cpus=8,threads=13,free=50820552,total=115970048
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SuggesterFSTTest, DateMathParserTest, PathHierarchyTokenizerFactoryTest, PluginInfoTest, SolrRequestParserTest, TestGroupingSearch, TestIBSimilarityFactory, ZkControllerTest, ResponseLogComponentTest, DirectSolrConnectionTest, BinaryUpdateRequestHandlerTest, LeaderElectionIntegrationTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderElectionIntegrationTest -Dtests.seed=141DB14FC9DA2771 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=pl_PL -Dtests.timezone=Australia/Broken_Hill -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR   0.00s J0 | LeaderElectionIntegrationTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([141DB14FC9DA2771]:0)
[junit4:junit4] Completed on J0 in 7223.30s, 1 test, 2 errors <<< FAILURES!

[...truncated 146 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:335: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:178: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/module-build.xml:61: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1169: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:837: There were test failures: 232 suites, 960 tests, 1 suite-level error, 1 error, 182 ignored (1 assumption)

Total time: 143 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 64bit/jrockit-jdk1.6.0_33-R28.2.4-4.1.0 -XnoOpt
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message