lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 2088 - Still Unstable!
Date Tue, 02 Oct 2018 23:46:49 GMT
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/2088/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove

Error Message:
No live SolrServers available to handle this request:[https://127.0.0.1:33819/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:33622/solr/MoveReplicaHDFSTest_failed_coll_true]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:33819/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:33622/solr/MoveReplicaHDFSTest_failed_coll_true]
	at __randomizedtesting.SeedInfo.seed([5BDB6A5C789C5E77:F116B9AECF4F8BA7]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:994)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
	at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:301)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33622/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard2
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	... 46 more




Build Log:
[...truncated 12524 lines...]
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/init-core-data-001
   [junit4]   2> 226005 WARN  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 226005 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 226007 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 226008 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 226008 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-001
   [junit4]   2> 226008 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 226008 INFO  (Thread-469) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 226008 INFO  (Thread-469) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 226010 ERROR (Thread-469) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 226115 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:48890
   [junit4]   2> 226117 INFO  (zkConnectionManagerCallback-620-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 226136 INFO  (jetty-launcher-617-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 226136 INFO  (jetty-launcher-617-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 226136 INFO  (jetty-launcher-617-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 226137 INFO  (jetty-launcher-617-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 226137 INFO  (jetty-launcher-617-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 226137 INFO  (jetty-launcher-617-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 226137 INFO  (jetty-launcher-617-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 226138 INFO  (jetty-launcher-617-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 226138 INFO  (jetty-launcher-617-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 226138 INFO  (jetty-launcher-617-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 226138 INFO  (jetty-launcher-617-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 226138 INFO  (jetty-launcher-617-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 226139 INFO  (jetty-launcher-617-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 226139 INFO  (jetty-launcher-617-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 226139 INFO  (jetty-launcher-617-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 226139 INFO  (jetty-launcher-617-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 226140 INFO  (jetty-launcher-617-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3b798ccd{/solr,null,AVAILABLE}
   [junit4]   2> 226140 INFO  (jetty-launcher-617-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@64399082{/solr,null,AVAILABLE}
   [junit4]   2> 226141 INFO  (jetty-launcher-617-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@400d3c96{SSL,[ssl, http/1.1]}{127.0.0.1:33819}
   [junit4]   2> 226141 INFO  (jetty-launcher-617-thread-3) [    ] o.e.j.s.Server Started @226337ms
   [junit4]   2> 226141 INFO  (jetty-launcher-617-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33819}
   [junit4]   2> 226141 ERROR (jetty-launcher-617-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 226141 INFO  (jetty-launcher-617-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 226141 INFO  (jetty-launcher-617-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 226141 INFO  (jetty-launcher-617-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 226141 INFO  (jetty-launcher-617-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 226141 INFO  (jetty-launcher-617-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-02T22:00:34.892Z
   [junit4]   2> 226141 INFO  (jetty-launcher-617-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1519c3f2{/solr,null,AVAILABLE}
   [junit4]   2> 226142 INFO  (jetty-launcher-617-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@f447c12{SSL,[ssl, http/1.1]}{127.0.0.1:44159}
   [junit4]   2> 226142 INFO  (jetty-launcher-617-thread-2) [    ] o.e.j.s.Server Started @226338ms
   [junit4]   2> 226142 INFO  (jetty-launcher-617-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44159}
   [junit4]   2> 226142 ERROR (jetty-launcher-617-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 226142 INFO  (jetty-launcher-617-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 226142 INFO  (jetty-launcher-617-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 226142 INFO  (jetty-launcher-617-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 226142 INFO  (jetty-launcher-617-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 226142 INFO  (jetty-launcher-617-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-02T22:00:34.893Z
   [junit4]   2> 226142 INFO  (jetty-launcher-617-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2aa6c6b4{/solr,null,AVAILABLE}
   [junit4]   2> 226143 INFO  (jetty-launcher-617-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@27474332{SSL,[ssl, http/1.1]}{127.0.0.1:33622}
   [junit4]   2> 226143 INFO  (jetty-launcher-617-thread-4) [    ] o.e.j.s.Server Started @226339ms
   [junit4]   2> 226143 INFO  (jetty-launcher-617-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33622}
   [junit4]   2> 226143 ERROR (jetty-launcher-617-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 226143 INFO  (jetty-launcher-617-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 226143 INFO  (jetty-launcher-617-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 226143 INFO  (jetty-launcher-617-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 226143 INFO  (jetty-launcher-617-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 226143 INFO  (jetty-launcher-617-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-02T22:00:34.894Z
   [junit4]   2> 226145 INFO  (jetty-launcher-617-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@11591924{SSL,[ssl, http/1.1]}{127.0.0.1:59659}
   [junit4]   2> 226145 INFO  (jetty-launcher-617-thread-1) [    ] o.e.j.s.Server Started @226341ms
   [junit4]   2> 226145 INFO  (jetty-launcher-617-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59659}
   [junit4]   2> 226146 ERROR (jetty-launcher-617-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 226146 INFO  (jetty-launcher-617-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 226146 INFO  (jetty-launcher-617-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 226146 INFO  (jetty-launcher-617-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 226146 INFO  (jetty-launcher-617-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 226146 INFO  (jetty-launcher-617-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-02T22:00:34.897Z
   [junit4]   2> 226148 INFO  (zkConnectionManagerCallback-626-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 226149 INFO  (zkConnectionManagerCallback-624-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 226149 INFO  (jetty-launcher-617-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 226149 INFO  (zkConnectionManagerCallback-622-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 226150 INFO  (zkConnectionManagerCallback-628-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 226150 INFO  (jetty-launcher-617-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 226150 INFO  (jetty-launcher-617-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 226151 INFO  (jetty-launcher-617-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 226154 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002b7e15ca0001, likely client has closed socket
   [junit4]   2> 226154 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002b7e15ca0002, likely client has closed socket
   [junit4]   2> 226768 INFO  (jetty-launcher-617-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48890/solr
   [junit4]   2> 226799 INFO  (zkConnectionManagerCallback-632-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 226861 DEBUG (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$340/470010957@28f87425
   [junit4]   2> 226947 INFO  (zkConnectionManagerCallback-634-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 227078 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 227078 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227078 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 227078 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 227079 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 227079 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 227082 DEBUG (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72105414569361414-127.0.0.1:59659_solr-n_0000000000
   [junit4]   2> 227082 INFO  (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59659_solr
   [junit4]   2> 227083 INFO  (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.c.Overseer Overseer (id=72105414569361414-127.0.0.1:59659_solr-n_0000000000) starting
   [junit4]   2> 227090 INFO  (zkConnectionManagerCallback-641-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 227090 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 227090 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 227091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 227091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 227091 INFO  (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48890/solr ready
   [junit4]   2> 227092 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 227093 INFO  (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59659_solr
   [junit4]   2> 227093 INFO  (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59659_solr
   [junit4]   2> 227093 DEBUG (OverseerAutoScalingTriggerThread-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 227095 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/autoscaling.json
   [junit4]   2> 227095 DEBUG (OverseerAutoScalingTriggerThread-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 227096 DEBUG (OverseerAutoScalingTriggerThread-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 227096 DEBUG (OverseerAutoScalingTriggerThread-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 227097 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227097 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227112 INFO  (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 227137 INFO  (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59659.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 227141 INFO  (jetty-launcher-617-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48890/solr
   [junit4]   2> 227142 INFO  (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59659.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 227142 INFO  (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59659.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 227142 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227142 INFO  (jetty-launcher-617-thread-1) [n:127.0.0.1:59659_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-001/node1/.
   [junit4]   2> 227142 INFO  (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 227143 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
   [junit4]   2> 227143 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 227143 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 227143 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 227143 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 227143 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
   [junit4]   2> 227144 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227144 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 227144 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue-work
   [junit4]   2> 227144 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 227155 INFO  (zkCallback-640-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 227156 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 227158 DEBUG (OverseerAutoScalingTriggerThread-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:59659_solr]
   [junit4]   2> 227158 DEBUG (OverseerAutoScalingTriggerThread-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 227159 DEBUG (OverseerAutoScalingTriggerThread-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 227159 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 227160 INFO  (zkConnectionManagerCallback-646-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 227160 DEBUG (ScheduledTrigger-1125-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 227161 DEBUG (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$340/470010957@33671335
   [junit4]   2> 227163 INFO  (zkConnectionManagerCallback-648-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 227168 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 227168 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227168 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 227169 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 227169 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 227169 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 227169 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 227171 DEBUG (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72105414569361417-127.0.0.1:33819_solr-n_0000000001
   [junit4]   2> 227171 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72105414569361414-127.0.0.1:59659_solr-n_0000000000
   [junit4]   2> 227171 DEBUG (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72105414569361414-127.0.0.1:59659_solr-n_0000000000 to know if I could be the leader
   [junit4]   2> 227171 DEBUG (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:33819_solr as DOWN
   [junit4]   2> 227172 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 227172 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 227173 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 227173 DEBUG (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:33819_solr"} current state version: 0
   [junit4]   2> 227173 DEBUG (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:33819_solr
   [junit4]   2> 227173 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33819_solr
   [junit4]   2> 227173 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227173 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227173 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227174 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227174 INFO  (zkCallback-633-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 227174 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227174 INFO  (zkCallback-640-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 227190 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227202 INFO  (zkCallback-647-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 227202 INFO  (zkConnectionManagerCallback-655-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 227203 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 227203 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227204 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 227204 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 227204 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 227204 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 227204 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48890/solr ready
   [junit4]   2> 227205 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 227228 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33819.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 227232 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33819.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 227232 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33819.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 227233 INFO  (jetty-launcher-617-thread-3) [n:127.0.0.1:33819_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-001/node4/.
   [junit4]   2> 227273 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 227274 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 227300 INFO  (jetty-launcher-617-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48890/solr
   [junit4]   2> 227302 INFO  (zkConnectionManagerCallback-660-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 227303 DEBUG (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$340/470010957@51cce95b
   [junit4]   2> 227305 INFO  (zkConnectionManagerCallback-662-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 227314 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 227314 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227314 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 227314 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 227314 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 227315 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 227315 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 227317 DEBUG (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72105414569361420-127.0.0.1:44159_solr-n_0000000002
   [junit4]   2> 227318 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72105414569361417-127.0.0.1:33819_solr-n_0000000001
   [junit4]   2> 227318 DEBUG (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72105414569361417-127.0.0.1:33819_solr-n_0000000001 to know if I could be the leader
   [junit4]   2> 227318 DEBUG (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:44159_solr as DOWN
   [junit4]   2> 227319 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 227319 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 227320 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44159_solr
   [junit4]   2> 227320 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 227320 DEBUG (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:44159_solr"} current state version: 0
   [junit4]   2> 227320 DEBUG (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:44159_solr
   [junit4]   2> 227321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 227321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227321 INFO  (zkCallback-640-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 227321 INFO  (zkCallback-633-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 227321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227322 INFO  (zkCallback-647-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 227322 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227322 INFO  (zkCallback-654-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 227323 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227323 INFO  (zkCallback-661-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 227340 INFO  (zkConnectionManagerCallback-669-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 227341 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 227341 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 227341 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 227341 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 227342 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 227342 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 227342 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48890/solr ready
   [junit4]   2> 227343 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 227355 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44159.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 227360 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44159.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 227360 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44159.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 227361 INFO  (jetty-launcher-617-thread-2) [n:127.0.0.1:44159_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-001/node2/.
   [junit4]   2> 227421 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 227422 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 228161 DEBUG (ScheduledTrigger-1125-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 228436 INFO  (jetty-launcher-617-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48890/solr
   [junit4]   2> 228438 INFO  (zkConnectionManagerCallback-674-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 228440 DEBUG (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$340/470010957@1dfbb377
   [junit4]   2> 228441 INFO  (zkConnectionManagerCallback-676-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 228453 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 228454 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228454 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 228454 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 228454 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 228455 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 228455 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 228458 DEBUG (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72105414569361423-127.0.0.1:33622_solr-n_0000000003
   [junit4]   2> 228458 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72105414569361420-127.0.0.1:44159_solr-n_0000000002
   [junit4]   2> 228458 DEBUG (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72105414569361420-127.0.0.1:44159_solr-n_0000000002 to know if I could be the leader
   [junit4]   2> 228459 DEBUG (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:33622_solr as DOWN
   [junit4]   2> 228460 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 228460 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 228460 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33622_solr
   [junit4]   2> 228460 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 228461 DEBUG (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:33622_solr"} current state version: 0
   [junit4]   2> 228461 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 228461 DEBUG (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:33622_solr
   [junit4]   2> 228461 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 228461 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 228461 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 228461 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 228461 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 228461 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 228461 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228462 INFO  (zkCallback-640-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 228462 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228462 INFO  (zkCallback-654-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 228462 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228462 INFO  (zkCallback-633-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 228462 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228462 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228463 INFO  (zkCallback-647-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 228463 INFO  (zkCallback-668-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 228464 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228464 INFO  (zkCallback-661-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 228464 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228465 INFO  (zkCallback-675-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 228485 INFO  (zkConnectionManagerCallback-683-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 228486 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 228487 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228487 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 228487 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 228487 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 228488 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 228488 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48890/solr ready
   [junit4]   2> 228489 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 228510 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33622.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 228515 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33622.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 228515 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33622.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 228516 INFO  (jetty-launcher-617-thread-4) [n:127.0.0.1:33622_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-001/node3/.
   [junit4]   2> 228542 INFO  (zkConnectionManagerCallback-686-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 228546 INFO  (zkConnectionManagerCallback-691-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 228547 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 228548 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 228548 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 228548 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 228549 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 228549 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 228550 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48890/solr ready
   [junit4]   2> 228562 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 228563 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 228578 INFO  (qtp143939268-2221) [n:127.0.0.1:44159_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 228579 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 228579 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 228579 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> 228580 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
   [junit4]   2> 228580 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 228581 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 228581 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
   [junit4]   2> 228581 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskQueue Returning topN elements: [/overseer/collection-queue-work/qn-0000000000, ]
   [junit4]   2> 228581 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Got 1 tasks from work-queue : [[org.apache.solr.cloud.OverseerTaskQueue$QueueEvent@2e684ecf]]
   [junit4]   2> 228582 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as running
   [junit4]   2> 228582 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{"operation":"overseerstatus"}
   [junit4]   2> 228583 DEBUG (OverseerThreadFactory-1134-thread-1-processing-n:127.0.0.1:59659_solr) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Runner processing /overseer/collection-queue-work/qn-0000000000
   [junit4]   2> 228583 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 1
   [junit4]   2> 228583 DEBUG (OverseerThreadFactory-1134-thread-1-processing-n:127.0.0.1:59659_solr) [n:127.0.0.1:59659_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler OverseerCollectionMessageHandler.processMessage : overseerstatus , {"operation":"overseerstatus"}
   [junit4]   2> 228583 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 228583 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 228583 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 228583 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 228583 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 99 elements. ExcludeSet: [/overseer/collection-queue-work/qn-0000000000],[]
   [junit4]   2> 228588 DEBUG (OverseerThreadFactory-1134-thread-1-processing-n:127.0.0.1:59659_solr) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Completed task:[/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 228589 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> 228589 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 228589 DEBUG (OverseerThreadFactory-1134-thread-1-processing-n:127.0.0.1:59659_solr) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as completed.
   [junit4]   2> 228589 DEBUG (OverseerThreadFactory-1134-thread-1-processing-n:127.0.0.1:59659_solr) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 228589 DEBUG (OverseerThreadFactory-1134-thread-1-processing-n:127.0.0.1:59659_solr) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 228589 DEBUG (OverseerThreadFactory-1134-thread-1-processing-n:127.0.0.1:59659_solr) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 228589 DEBUG (OverseerThreadFactory-1134-thread-1-processing-n:127.0.0.1:59659_solr) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 228589 DEBUG (zkCallback-661-thread-1) [    ] o.a.s.c.OverseerTaskQueue NodeDataChanged fired on path /overseer/collection-queue-work/qnr-0000000000 state SyncConnected latchEventType null
   [junit4]   2> 228590 DEBUG (OverseerThreadFactory-1134-thread-1-processing-n:127.0.0.1:59659_solr) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{leader=127.0.0.1:59659_solr,overseer_queue_size=0,overseer_work_queue_size=0,overseer_collection_queue_size=2,overseer_operations={am_i_leader={requests=5,errors=0,avgRequestsPerSecond=3.3448836312777055,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.47512472965065333,medianRequestTime=0.553621,75thPcRequestTime=0.632183,95thPcRequestTime=0.792284,99thPcRequestTime=0.792284,999thPcRequestTime=0.792284},downnode={requests=3,errors=0,avgRequestsPerSecond=2.121618755591831,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.026907061330503225,medianRequestTime=0.026123,75thPcRequestTime=0.029141,95thPcRequestTime=0.029141,99thPcRequestTime=0.029141,999thPcRequestTime=0.029141}},collection_operations={am_i_leader={requests=3,errors=0,avgRequestsPerSecond=2.0074576771662653,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.5489708470104767,medianRequestTime=0.662243,75thPcRequestTime=0.680821,95thPcRequestTime=0.680821,99thPcRequestTime=0.680821,999thPcRequestTime=0.680821},overseerstatus={requests=0,errors=0,avgRequestsPerSecond=0.0,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.0,medianRequestTime=0.0,75thPcRequestTime=0.0,95thPcRequestTime=0.0,99thPcRequestTime=0.0,999thPcRequestTime=0.0}},overseer_queue={},overseer_internal_queue={peek={avgRequestsPerSecond=0.6928643418116859,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.484548,medianRequestTime=0.484548,75thPcRequestTime=0.484548,95thPcRequestTime=0.484548,99thPcRequestTime=0.484548,999thPcRequestTime=0.484548}},collection_queue={peektopn_wait2000={avgRequestsPerSecond=0.6928116226443619,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=1438.7841,medianRequestTime=1438.7841,75thPcRequestTime=1438.7841,95thPcRequestTime=1438.7841,99thPcRequestTime=1438.7841,999thPcRequestTime=1438.7841}}}
   [junit4]   2> 228589 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 228590 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
   [junit4]   2> 228590 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 228591 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 228593 INFO  (qtp143939268-2221) [n:127.0.0.1:44159_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=14
   [junit4]   2> 229042 WARN  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 229162 DEBUG (ScheduledTrigger-1125-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 230163 DEBUG (ScheduledTrigger-1125-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 230312 WARN  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 230487 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log Logging to org.apache.logging.slf4j.Log4jLogger@6284db91 via org.mortbay.log.Slf4jLog
   [junit4]   2> 230512 WARN  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 230584 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
   [junit4]   2> 230585 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
   [junit4]   2> 230585 INFO  (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 230586 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 230586 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 230586 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 230586 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 230586 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
   [junit4]   2> 230845 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 230887 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_serv1.vm3_49073_hdfs____g9mkrs/webapp
   [junit4]   2> 231165 DEBUG (ScheduledTrigger-1125-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 231893 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@serv1-vm3:49073
   [junit4]   2> 232166 DEBUG (ScheduledTrigger-1125-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 232586 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
   [junit4]   2> 232588 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
   [junit4]   2> 232588 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 232588 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 232588 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 232588 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 232588 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
   [junit4]   2> 232886 WARN  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 232892 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 232906 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_37002_datanode____.70o1cb/webapp
   [junit4]   2> 233166 DEBUG (ScheduledTrigger-1125-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 233379 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37002
   [junit4]   2> 233712 WARN  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 233713 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 233724 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36889_datanode____szvg0t/webapp
   [junit4]   2> 234166 DEBUG (ScheduledTrigger-1125-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 234210 ERROR (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to serv1-vm3/127.0.0.1:36831) [    ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
   [junit4]   2> 234221 INFO  (SUITE-MoveReplicaHDFSTest-seed#[5BDB6A5C789C5E77]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36889
   [junit4]   2> 234378 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x297a69a2c0e53: from storage DS-77db7373-33bc-4eb1-8b68-59addaa882f0 node DatanodeRegistration(127.0.0.1:51448, datanodeUuid=e2c33351-230b-455f-b61b-97af2fe6d6bc, infoPort=42955, infoSecurePort=0, ipcPort=50150, storageInfo=lv=-56;cid=testClusterID;nsid=1532662712;c=0), blocks: 0, hasStaleStorage: true, processing time: 2 msecs
   [junit4]   2> 234378 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x297a69a2c0e53: from storage DS-1af2673f-f899-4f1c-89c1-0d8771964e92 node DatanodeRegistration(127.0.0.1:51448, datanodeUuid=e2c33351-230b-455f-b61b-97af2fe6d6bc, infoPort=42955, infoSecurePort=0, ipcPort=50150, storageInfo=lv=-56;cid=testClusterID;nsid=1532662712;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 234588 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
   [junit4]   2> 234589 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
   [junit4]   2> 234589 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 234589 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 234589 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 234589 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 234589 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
   [junit4]   2> 235170 DEBUG (ScheduledTrigger-1125-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 235229 ERROR (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to serv1-vm3/127.0.0.1:36831) [    ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
   [junit4]   2> 235239 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x297a6ce4f931c: from storage DS-cefea033-6b81-4515-9307-b8b3d73544e1 node DatanodeRegistration(127.0.0.1:41734, datanodeUuid=acac84d3-814e-4abc-aaaa-01851e97f760, infoPort=47244, infoSecurePort=0, ipcPort=48891, storageInfo=lv=-56;cid=testClusterID;nsid=1532662712;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 235239 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x297a6ce4f931c: from storage DS-9b05c656-d0c2-4e3d-9b69-c52cdf60617b node DatanodeRegistration(127.0.0.1:41734, datanodeUuid=acac84d3-814e-4abc-aaaa-01851e97f760, infoPort=47244, infoSecurePort=0, ipcPort=48891, storageInfo=lv=-56;cid=testClusterID;nsid=1532662712;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 235502 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNormalMove
   [junit4]   2> 235503 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 235504 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235504 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 235504 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 235504 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 235505 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 235506 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 235506 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 235506 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 235506 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 235506 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 235506 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 235506 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 235506 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 235506 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 235507 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235507 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235507 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235507 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235507 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235508 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235508 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235508 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235509 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235509 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 235511 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 235512 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 235512 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 235512 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 235512 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ade22bc{/solr,null,AVAILABLE}
   [junit4]   2> 235513 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@35842749{SSL,[ssl, http/1.1]}{127.0.0.1:41940}
   [junit4]   2> 235513 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.e.j.s.Server Started @235713ms
   [junit4]   2> 235513 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41940}
   [junit4]   2> 235513 ERROR (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 235513 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 235513 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 235513 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 235513 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 235513 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-02T22:00:44.264Z
   [junit4]   2> 235516 INFO  (zkConnectionManagerCallback-695-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 235517 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 235839 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48890/solr
   [junit4]   2> 235841 INFO  (zkConnectionManagerCallback-699-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 235863 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$340/470010957@616bfa3f
   [junit4]   2> 235885 INFO  (zkConnectionManagerCallback-701-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 235894 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 235894 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235894 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 235895 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 235895 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 235895 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 235898 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 235900 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72105414569361429-127.0.0.1:41940_solr-n_0000000004
   [junit4]   2> 235900 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72105414569361423-127.0.0.1:33622_solr-n_0000000003
   [junit4]   2> 235900 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72105414569361423-127.0.0.1:33622_solr-n_0000000003 to know if I could be the leader
   [junit4]   2> 235901 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:41940_solr as DOWN
   [junit4]   2> 235902 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 235902 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 235902 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 235902 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41940_solr
   [junit4]   2> 235905 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 235906 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235906 INFO  (zkCallback-675-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235907 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235907 DEBUG (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:41940_solr"} current state version: 0
   [junit4]   2> 235907 INFO  (zkCallback-633-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235907 DEBUG (OverseerStateUpdate-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:41940_solr
   [junit4]   2> 235908 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235908 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235908 INFO  (zkCallback-640-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235909 INFO  (zkCallback-690-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235910 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235910 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235910 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235910 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235910 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235910 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235910 INFO  (zkCallback-661-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235910 INFO  (zkCallback-654-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235939 INFO  (zkCallback-682-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235939 INFO  (zkCallback-690-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235939 INFO  (zkCallback-647-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235939 INFO  (zkCallback-668-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235939 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235940 INFO  (zkCallback-700-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 235958 INFO  (zkConnectionManagerCallback-708-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 235959 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 235959 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 235959 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 235960 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 235960 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 235960 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 235961 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48890/solr ready
   [junit4]   2> 235961 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 235982 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41940.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 235993 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41940.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 235993 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41940.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70ff946
   [junit4]   2> 235994 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [n:127.0.0.1:41940_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_5BDB6A5C789C5E77-001/tempDir-001/node5/.
   [junit4]   2> 236009 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 236030 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 236046 INFO  (zkConnectionManagerCallback-711-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 236047 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[5BDB6A5C789C5E77]) [    ] o.a.s.c.MoveReplicaTest total_jettys: 5
   [junit4]   2> 236063 INFO  (qtp1852417139-2216) [n:127.0.0.1:33622_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&maxShardsPerNode=2&autoAddReplicas=false&name=MoveReplicaHDFSTest_coll_false&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 236064 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 236064 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000002
   [junit4]   2> 236064 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 236065 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000002
   [junit4]   2> 236065 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 236065 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 236065 DEBUG (OverseerCollectionConfigSetProcessor-72105414569361414-127.0.0.1:59659_solr-n_0000000000) [n:127.0.0.1:59659_solr    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000002
 

[...truncated too long message...]

 Apache Ivy 2.4.0 - 20141213170938 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/top-level-ivy-settings.xml

install-junit4-taskdef:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

resolve-groovy:
[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.15 in public
[ivy:cachepath] :: resolution report :: resolve 17ms :: artifacts dl 0ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |
	---------------------------------------------------------------------

-init-totals:

-test:
    [mkdir] Created dir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test
[junit4:pickseed] Seed property 'tests.seed' already defined: 5BDB6A5C789C5E77
    [mkdir] Created dir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/temp
   [junit4] <JUnit4> says g'day! Master seed: 5BDB6A5C789C5E77
   [junit4] Executing 5 suites with 2 JVMs.
   [junit4] 
   [junit4] Started J1 PID(7395@serv1-vm3).
   [junit4] Started J0 PID(7396@serv1-vm3).
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
   [junit4] OK      25.0s J0 | MoveReplicaHDFSTest.testNormalMove
   [junit4] OK      15.3s J0 | MoveReplicaHDFSTest.testFailedMove
   [junit4] OK      14.8s J0 | MoveReplicaHDFSTest.testNormalFailedMove
   [junit4] OK      25.3s J0 | MoveReplicaHDFSTest.test
   [junit4] Completed [1/5] on J0 in 109.98s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
   [junit4] OK      28.4s J1 | MoveReplicaHDFSTest.testNormalMove
   [junit4] OK      15.3s J1 | MoveReplicaHDFSTest.testFailedMove
   [junit4] OK      14.7s J1 | MoveReplicaHDFSTest.testNormalFailedMove
   [junit4] OK      26.5s J1 | MoveReplicaHDFSTest.test
   [junit4] Completed [2/5] on J1 in 114.51s, 4 tests
   [junit4] 
   [junit4] Duplicate suite name used with XML reports: org.apache.solr.cloud.MoveReplicaHDFSTest. This may confuse tools that process XML reports. Set 'ignoreDuplicateSuites' to true to skip this message.
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
   [junit4] OK      26.4s J0 | MoveReplicaHDFSTest.testNormalMove
   [junit4] OK      15.9s J0 | MoveReplicaHDFSTest.testFailedMove
   [junit4] OK      13.3s J0 | MoveReplicaHDFSTest.testNormalFailedMove
   [junit4] OK      24.7s J0 | MoveReplicaHDFSTest.test
   [junit4] Completed [3/5] on J0 in 96.02s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
   [junit4] OK      23.0s J1 | MoveReplicaHDFSTest.testNormalMove
   [junit4] OK      15.3s J1 | MoveReplicaHDFSTest.testFailedMove
   [junit4] OK      14.0s J1 | MoveReplicaHDFSTest.testNormalFailedMove
   [junit4] OK      23.4s J1 | MoveReplicaHDFSTest.test
   [junit4] Completed [4/5] on J1 in 92.16s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
   [junit4] OK      20.5s J0 | MoveReplicaHDFSTest.testNormalMove
   [junit4] OK      13.7s J0 | MoveReplicaHDFSTest.testFailedMove
   [junit4] OK      14.1s J0 | MoveReplicaHDFSTest.testNormalFailedMove
   [junit4] OK      23.9s J0 | MoveReplicaHDFSTest.test
   [junit4] Completed [5/5] on J0 in 87.70s, 4 tests
   [junit4] 
   [junit4] JVM J0:     0.67 ..   295.60 =   294.92s
   [junit4] JVM J1:     0.67 ..   208.56 =   207.88s
   [junit4] Execution time total: 4 minutes 55 seconds
   [junit4] Tests summary: 5 suites, 20 tests
   [junit4] Could not remove temporary path: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp])
   [junit4] Could not remove temporary path: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp])
     [echo] 5 slowest tests:
[junit4:tophints] 1266.07s | org.apache.solr.cloud.MultiThreadedOCPTest
[junit4:tophints] 376.62s | org.apache.solr.cloud.cdcr.CdcrWithNodesRestartsTest
[junit4:tophints] 325.16s | org.apache.solr.cloud.api.collections.ShardSplitTest
[junit4:tophints] 245.63s | org.apache.solr.cloud.cdcr.CdcrOpsAndBoundariesTest
[junit4:tophints] 123.84s | org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest

-check-totals:

test-nocompile:

BUILD SUCCESSFUL
Total time: 4 minutes 58 seconds
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.MoveReplicaHDFSTest
[repro] Exiting with code 0
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2845)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2724)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2705)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3084)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to Solaris VBOX
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1070)
		at hudson.FilePath.act(FilePath.java:1059)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2703)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1831)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3086)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1072)
	at hudson.FilePath.act(FilePath.java:1059)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2703)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1831)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/*_pid*.log". Configuration error?
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=2201b6526635597aba170bc8f39a4bb438d16b93, workspace=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris
[WARNINGS] Computing warning deltas based on reference build #2087
Recording test results
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

Mime
View raw message