lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0-ea-b106) - Build # 3269 - Still Failing!
Date Wed, 18 Sep 2013 01:48:31 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3269/
Java: 64bit/jdk1.8.0-ea-b106 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionTest

Error Message:
8 threads leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest:     1) Thread[id=5206, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)    2) Thread[id=5211, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)    3) Thread[id=5197, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)    4) Thread[id=5207, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)    5) Thread[id=5198, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)    6) Thread[id=5203, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)    7) Thread[id=5202, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)    8) Thread[id=5210, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest: 
   1) Thread[id=5206, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   2) Thread[id=5211, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   3) Thread[id=5197, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   4) Thread[id=5207, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   5) Thread[id=5198, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   6) Thread[id=5203, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   7) Thread[id=5202, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   8) Thread[id=5210, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
	at __randomizedtesting.SeedInfo.seed([B92AD3A624B4B197]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=5206, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)    2) Thread[id=5197, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)    3) Thread[id=5202, name=Thread-2087-SendThread(127.0.0.1:53623), state=TIMED_WAITING, group=TGRP-LeaderElectionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)    4) Thread[id=5210, name=Thread-2087-SendThread(127.0.0.1:53623), state=TIMED_WAITING, group=TGRP-LeaderElectionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=5206, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   2) Thread[id=5197, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   3) Thread[id=5202, name=Thread-2087-SendThread(127.0.0.1:53623), state=TIMED_WAITING, group=TGRP-LeaderElectionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   4) Thread[id=5210, name=Thread-2087-SendThread(127.0.0.1:53623), state=TIMED_WAITING, group=TGRP-LeaderElectionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
	at __randomizedtesting.SeedInfo.seed([B92AD3A624B4B197]:0)




Build Log:
[...truncated 10205 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderElectionTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-LeaderElectionTest-1379468786291
   [junit4]   2> 1295774 T5155 oas.SolrTestCaseJ4.setUp ###Starting testCancelElection
   [junit4]   2> 1295774 T5155 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1295775 T5156 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1295875 T5155 oasc.ZkTestServer.run start zk server on port:53599
   [junit4]   2> 1295877 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1295884 T5162 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52193822 name:ZooKeeperConnection Watcher:127.0.0.1:53599 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1295884 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1295889 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1295891 T5164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cdecc11 name:ZooKeeperConnection Watcher:127.0.0.1:53599 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1295891 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1295891 T5155 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1295954 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1295956 T5166 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58e15352 name:ZooKeeperConnection Watcher:127.0.0.1:53599/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1295956 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1295957 T5155 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/slice1/election
   [junit4]   2> 1295966 T5155 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/slice1
   [junit4]   2> 1295971 T5155 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1297978 T5166 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/slice1
   [junit4]   2> 1298987 T5155 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53599 53599
   [junit4]   2> 1301238 T5155 oas.SolrTestCaseJ4.tearDown ###Ending testCancelElection
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1301242 T5155 oas.SolrTestCaseJ4.setUp ###Starting testBasic
   [junit4]   2> 1301243 T5155 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1301245 T5167 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1301344 T5155 oasc.ZkTestServer.run start zk server on port:53611
   [junit4]   2> 1301357 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1301364 T5173 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a0259c name:ZooKeeperConnection Watcher:127.0.0.1:53611 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1301365 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1301394 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1301396 T5175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c34df1d name:ZooKeeperConnection Watcher:127.0.0.1:53611 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1301396 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1301396 T5155 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1301404 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1301406 T5177 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e9d620a name:ZooKeeperConnection Watcher:127.0.0.1:53611/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1301407 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1301408 T5155 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 1301419 T5155 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 1301427 T5155 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1301439 T5155 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53611 53611
   [junit4]   2> 1303691 T5155 oas.SolrTestCaseJ4.tearDown ###Ending testBasic
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1303714 T5155 oas.SolrTestCaseJ4.setUp ###Starting testStressElection
   [junit4]   2> 1303714 T5155 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1303717 T5178 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1303816 T5155 oasc.ZkTestServer.run start zk server on port:53623
   [junit4]   2> 1303818 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1303827 T5184 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a477f85 name:ZooKeeperConnection Watcher:127.0.0.1:53623 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1303828 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1303846 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1303849 T5186 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70018be7 name:ZooKeeperConnection Watcher:127.0.0.1:53623 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1303849 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1303850 T5155 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1303857 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1303862 T5188 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@65992adf name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1303862 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1303876 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1303882 T5191 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3878d3fa name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1303882 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1303884 T5192 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1303901 T5192 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1303909 T5192 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1305893 T5193 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1311144 T5193 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1311146 T5179 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1412ebf59890003, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 
   [junit4]   2> 1311147 T5198 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13913bcd name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1311147 T5198 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
   [junit4]   2> 1311150 T5193 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1311154 T5203 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17aebd67 name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1311154 T5203 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
   [junit4]   2> 1311156 T5207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7961b3da name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1311157 T5207 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
   [junit4]   2> 1311157 T5193 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1311163 T5211 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63b4a068 name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1311163 T5211 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
   [junit4]   2> 1311261 T5191 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3878d3fa name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 1311261 T5191 oascc.ConnectionManager.process zkClient has disconnected
   [junit4]   2> 1311269 T5155 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53623 53623
   [junit4]   2> 1313605 T5155 oas.SolrTestCaseJ4.tearDown ###Ending testStressElection
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1313619 T5155 oas.SolrTestCaseJ4.setUp ###Starting testElection
   [junit4]   2> 1313620 T5155 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1313621 T5212 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1313715 T5207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7961b3da name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 1313716 T5207 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
   [junit4]   2> 1313716 T5211 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63b4a068 name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 1313716 T5211 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
   [junit4]   2> 1313716 T5198 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13913bcd name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 1313717 T5198 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
   [junit4]   2> 1313717 T5203 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17aebd67 name:ZooKeeperConnection Watcher:127.0.0.1:53623/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 1313717 T5203 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
   [junit4]   2> 1313731 T5155 oasc.ZkTestServer.run start zk server on port:53653
   [junit4]   2> 1313735 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313747 T5218 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5dc4519c name:ZooKeeperConnection Watcher:127.0.0.1:53653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313748 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313781 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313792 T5220 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@269cb931 name:ZooKeeperConnection Watcher:127.0.0.1:53653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313792 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313792 T5155 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1313799 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313805 T5222 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6714a9ed name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313805 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313806 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313815 T5225 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47e10b20 name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313815 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313816 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313827 T5228 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38d6eaf1 name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313827 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313830 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313834 T5231 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7bc099eb name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313834 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313835 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313844 T5234 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39d91cbf name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313845 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313847 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313850 T5237 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76aeb8af name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313850 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313853 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313855 T5240 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@792055e9 name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313855 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313858 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313860 T5243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d094e7 name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313860 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313862 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313864 T5246 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@779d5c66 name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313864 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313866 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313868 T5249 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74f5e9fa name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313868 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313870 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313875 T5252 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f016590 name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313875 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313876 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313879 T5255 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d74af3b name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313879 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313881 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313883 T5258 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1017ac9f name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313883 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313885 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313889 T5261 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4bf63d5e name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313889 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313890 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313894 T5264 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f5fec55 name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313894 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313895 T5155 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1313899 T5267 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@677a9ff2 name:ZooKeeperConnection Watcher:127.0.0.1:53653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1313899 T5155 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1313903 T5226 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313903 T5232 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313905 T5238 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313907 T5244 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313908 T5250 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313909 T5256 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313912 T5262 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313917 T5223 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313931 T5229 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313931 T5235 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313932 T5241 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313933 T5247 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313933 T5253 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313934 T5259 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1313934 T5265 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1314063 T5223 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1314113 T5223 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1314154 T5228 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1314710 T5246 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1315272 T5240 oasc.LeaderElector.checkIfIamLeader WARN Failed setting watch org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election/90404867550871562-7-n_0000000002
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1151)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:252)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:249)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:249)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:117)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1315281 T5237 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1315867 T5155 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53653 53653
   [junit4]   2> 1316077 T5202 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1318346 T5155 oas.SolrTestCaseJ4.tearDown ###Ending testElection
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1318347 T5155 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 22620 T5154 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 17 leaked thread(s).
   [junit4]   2> 1317290 T5267 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1318391 T5267 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:184)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:392)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:361)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:348)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:95)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1317284 T5261 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1318392 T5261 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:184)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:392)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:361)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:348)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:95)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1317279 T5255 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1318392 T5255 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:184)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:392)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:361)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:348)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:95)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1317275 T5249 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1318393 T5249 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:184)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:392)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:361)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:348)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:95)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1317270 T5243 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1318393 T5243 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:184)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:392)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:361)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:348)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:95)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1316710 T5231 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1318399 T5231 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:184)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:392)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:361)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:348)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:95)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1316706 T5234 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1318400 T5234 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:184)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:181)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:392)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:361)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:348)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:95)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1316704 T5240 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:148)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1316329 T5210 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890007 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1316196 T5197 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1316183 T5206 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1321092 T5197 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1321470 T5202 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1321509 T5210 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890007 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1321691 T5206 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1324149 T5206 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1324263 T5197 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1324548 T5202 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1324759 T5210 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890007 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1326616 T5206 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1326924 T5210 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890007 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1326963 T5197 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1327160 T5202 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1329031 T5210 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890007 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1329090 T5206 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1329866 T5197 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1330228 T5202 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1331575 T5206 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1331612 T5210 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890007 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1332366 T5202 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1332388 T5197 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1334263 T5206 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1334342 T5210 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890007 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1334666 T5202 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1335218 T5197 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1336290 T5206 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1336403 T5210 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890007 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1336889 T5202 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1337260 T5197 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 42695 T5154 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 8 threads leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest: 
   [junit4]   2> 	   1) Thread[id=5206, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 	   2) Thread[id=5211, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 	   3) Thread[id=5197, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 	   4) Thread[id=5207, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 	   5) Thread[id=5198, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 	   6) Thread[id=5203, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 	   7) Thread[id=5202, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 	   8) Thread[id=5210, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]   2> 	        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 42697 T5154 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=5206, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	   2) Thread[id=5211, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	   3) Thread[id=5197, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	   4) Thread[id=5207, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	   5) Thread[id=5198, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	   6) Thread[id=5203, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	   7) Thread[id=5202, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	   8) Thread[id=5210, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]   2> 1338467 T5211 oaz.ClientCnxn$EventThread.run ERROR Event thread exiting due to interruption java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2012)
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2046)
   [junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 
   [junit4]   2> 1338468 T5203 oaz.ClientCnxn$EventThread.run ERROR Event thread exiting due to interruption java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2012)
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2046)
   [junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 
   [junit4]   2> 1338468 T5198 oaz.ClientCnxn$EventThread.run ERROR Event thread exiting due to interruption java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2012)
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2046)
   [junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 
   [junit4]   2> 1338469 T5207 oaz.ClientCnxn$EventThread.run ERROR Event thread exiting due to interruption java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2012)
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2046)
   [junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 
   [junit4]   2> 1338593 T5210 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890007 for server null, unexpected error, closing socket connection and attempting reconnect java.nio.channels.ClosedByInterruptException
   [junit4]   2> 	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1338753 T5206 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890006 for server null, unexpected error, closing socket connection and attempting reconnect java.nio.channels.ClosedByInterruptException
   [junit4]   2> 	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1338972 T5206 oaz.ClientCnxn$SendThread.run WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
   [junit4]   2> 
   [junit4]   2> 1338973 T5210 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 
   [junit4]   2> 1339150 T5197 oaz.ClientCnxn$SendThread.run WARN Session 0x1412ebf59890004 for server null, unexpected error, closing socket connection and attempting reconnect java.nio.channels.ClosedByInterruptException
   [junit4]   2> 	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
   [junit4]   2>

[...truncated too long message...]

eper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]    >    2) Thread[id=5211, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]    >    3) Thread[id=5197, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]    >    4) Thread[id=5207, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]    >    5) Thread[id=5198, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]    >    6) Thread[id=5203, name=Thread-2087-EventThread, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]    >    7) Thread[id=5202, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]    >    8) Thread[id=5210, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B92AD3A624B4B197]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=5206, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]    >    2) Thread[id=5197, name=Thread-2087-SendThread(127.0.0.1:53623), state=RUNNABLE, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]    >         at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]    >    3) Thread[id=5202, name=Thread-2087-SendThread(127.0.0.1:53623), state=TIMED_WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]    >    4) Thread[id=5210, name=Thread-2087-SendThread(127.0.0.1:53623), state=TIMED_WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B92AD3A624B4B197]:0)
   [junit4] Completed in 45.76s, 4 tests, 2 errors <<< FAILURES!

[...truncated 474 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:396: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:376: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:455: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1230: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:873: There were test failures: 326 suites, 1448 tests, 2 suite-level errors, 721 ignored (6 assumptions)

Total time: 47 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-ea-b106 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message