lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-4.x-Linux (32bit/ibm-j9-jdk6) - Build # 7319 - Failure!
Date Tue, 10 Sep 2013 23:34:05 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/7319/
Java: 32bit/ibm-j9-jdk6 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

1 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
Server at http://127.0.0.1:44691/qwrk/i/onenodecollectioncore returned non ok status:404, message:Can not find: /qwrk/i/onenodecollectioncore/update

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:44691/qwrk/i/onenodecollectioncore returned non ok status:404, message:Can not find: /qwrk/i/onenodecollectioncore/update
	at __randomizedtesting.SeedInfo.seed([AD1928ECDAB6619D:2CFFA6F4ADE901A1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
	at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:738)




Build Log:
[...truncated 9453 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
   [junit4]   2> 801276 T2925 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /qwrk/i
   [junit4]   2> 801280 T2925 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1378855174152
   [junit4]   2> 801281 T2925 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 801282 T2926 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 801382 T2925 oasc.ZkTestServer.run start zk server on port:56132
   [junit4]   2> 801385 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 801444 T2932 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ebc7ebc name:ZooKeeperConnection Watcher:127.0.0.1:56132 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 801445 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 801447 T2925 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 801454 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 801487 T2934 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f424f42 name:ZooKeeperConnection Watcher:127.0.0.1:56132/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 801488 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 801488 T2925 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 801496 T2925 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 801504 T2925 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 801510 T2925 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 801518 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 801519 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 801530 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 801531 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 801539 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 801540 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 801548 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 801549 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 801557 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 801558 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 801566 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 801567 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 801575 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 801575 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 801583 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 801584 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 801592 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 801593 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 801602 T2925 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 801603 T2925 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 801784 T2925 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 801787 T2925 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33852
   [junit4]   2> 801788 T2925 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 801789 T2925 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 801790 T2925 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1378855174484
   [junit4]   2> 801790 T2925 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1378855174484/'
   [junit4]   2> 801804 T2925 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1378855174484/solr.xml
   [junit4]   2> 801848 T2925 oasc.CoreContainer.<init> New CoreContainer 1836477814
   [junit4]   2> 801849 T2925 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1378855174484/]
   [junit4]   2> 801850 T2925 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 801851 T2925 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 801852 T2925 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 801853 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 801853 T2925 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 801854 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 801855 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 801856 T2925 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 801857 T2925 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 801857 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 801864 T2925 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 801865 T2925 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 801866 T2925 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56132/solr
   [junit4]   2> 801867 T2925 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 801869 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 801871 T2945 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47054705 name:ZooKeeperConnection Watcher:127.0.0.1:56132 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 801872 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 801877 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 801883 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 801886 T2947 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fb52fb5 name:ZooKeeperConnection Watcher:127.0.0.1:56132/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 801887 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 801891 T2925 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 801908 T2925 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 801920 T2925 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 801927 T2925 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33852_qwrk%2Fi
   [junit4]   2> 801930 T2925 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33852_qwrk%2Fi
   [junit4]   2> 801942 T2925 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 801957 T2925 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 801965 T2925 oasc.Overseer.start Overseer (id=90364652693618691-127.0.0.1:33852_qwrk%2Fi-n_0000000000) starting
   [junit4]   2> 801986 T2925 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 801998 T2949 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 802000 T2925 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 802009 T2925 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 802016 T2925 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 802028 T2948 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 802030 T2950 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 802031 T2950 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 802034 T2950 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 803544 T2948 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 803546 T2948 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33852/qwrk/i",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:33852_qwrk%2Fi",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 803547 T2948 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 803548 T2948 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 803560 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 804034 T2950 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 804036 T2950 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1378855174484/collection1
   [junit4]   2> 804037 T2950 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 804042 T2950 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 804043 T2950 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 804052 T2950 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1378855174484/collection1/'
   [junit4]   2> 804055 T2950 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1378855174484/collection1/lib/classes/' to classloader
   [junit4]   2> 804057 T2950 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1378855174484/collection1/lib/README' to classloader
   [junit4]   2> 804108 T2950 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
   [junit4]   2> 804159 T2950 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 804166 T2950 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 804175 T2950 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 804633 T2950 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 804634 T2950 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 804634 T2950 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 804640 T2950 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 804643 T2950 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 804703 T2950 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 804710 T2950 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 804719 T2950 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 804724 T2950 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 804724 T2950 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 804725 T2950 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 804730 T2950 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 804731 T2950 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 804732 T2950 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 804733 T2950 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1378855174484/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/control/data/
   [junit4]   2> 804735 T2950 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@57da57da
   [junit4]   2> 804737 T2950 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/control/data
   [junit4]   2> 804738 T2950 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/control/data/index/
   [junit4]   2> 804739 T2950 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 804741 T2950 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/control/data/index
   [junit4]   2> 804742 T2950 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=49.9873046875, floorSegmentMB=1.6240234375, forceMergeDeletesPctAllowed=9.768604114962432, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8977219860458391
   [junit4]   2> 804744 T2950 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5aa05aa lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9680968),segFN=segments_1,generation=1}
   [junit4]   2> 804745 T2950 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 804752 T2950 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 804753 T2950 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 804753 T2950 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 804754 T2950 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 804755 T2950 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 804755 T2950 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 804756 T2950 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 804756 T2950 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 804757 T2950 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 804758 T2950 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 804758 T2950 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 804759 T2950 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 804760 T2950 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 804761 T2950 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 804761 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 804763 T2950 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 804767 T2950 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 804767 T2950 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 804768 T2950 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=36.4296875, floorSegmentMB=1.47265625, forceMergeDeletesPctAllowed=14.476735145232706, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 804769 T2950 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5aa05aa lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9680968),segFN=segments_1,generation=1}
   [junit4]   2> 804770 T2950 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 804770 T2950 oass.SolrIndexSearcher.<init> Opening Searcher@12451245 main
   [junit4]   2> 804775 T2951 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12451245 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 804780 T2950 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 804780 T2950 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33852/qwrk/i collection:control_collection shard:shard1
   [junit4]   2> 804781 T2950 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 804790 T2950 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 804847 T2950 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 804855 T2950 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 804856 T2950 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 804857 T2950 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:33852/qwrk/i/collection1/
   [junit4]   2> 804858 T2950 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 804859 T2950 oasc.SyncStrategy.syncToMe http://127.0.0.1:33852/qwrk/i/collection1/ has no replicas
   [junit4]   2> 804871 T2950 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:33852/qwrk/i/collection1/ shard1
   [junit4]   2> 804872 T2950 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 805083 T2948 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 805109 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 805145 T2950 oasc.ZkController.register We are http://127.0.0.1:33852/qwrk/i/collection1/ and leader is http://127.0.0.1:33852/qwrk/i/collection1/
   [junit4]   2> 805146 T2950 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33852/qwrk/i
   [junit4]   2> 805147 T2950 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 805148 T2950 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 805150 T2950 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 805159 T2950 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 805164 T2925 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 805166 T2925 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 805169 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 805182 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:
   [junit4]   2> 805186 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 805189 T2954 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18291829 name:ZooKeeperConnection Watcher:127.0.0.1:56132/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 805190 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 805198 T2925 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 805214 T2925 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
   [junit4]   2> 805419 T2925 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 805421 T2925 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47892
   [junit4]   2> 805422 T2925 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 805423 T2925 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 805424 T2925 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1378855178087
   [junit4]   2> 805425 T2925 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1378855178087/'
   [junit4]   2> 805468 T2925 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1378855178087/solr.xml
   [junit4]   2> 805511 T2925 oasc.CoreContainer.<init> New CoreContainer 216009952
   [junit4]   2> 805512 T2925 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1378855178087/]
   [junit4]   2> 805514 T2925 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 805515 T2925 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 805515 T2925 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 805516 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 805517 T2925 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 805518 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 805518 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 805519 T2925 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 805520 T2925 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 805521 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 805527 T2925 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 805528 T2925 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 805530 T2925 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56132/solr
   [junit4]   2> 805531 T2925 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 805532 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 805534 T2965 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42ae42ae name:ZooKeeperConnection Watcher:127.0.0.1:56132 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 805535 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 805539 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 805546 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 805548 T2967 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66646664 name:ZooKeeperConnection Watcher:127.0.0.1:56132/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 805549 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 805564 T2925 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 806577 T2925 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47892_qwrk%2Fi
   [junit4]   2> 806582 T2925 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47892_qwrk%2Fi
   [junit4]   2> 806600 T2954 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 806614 T2967 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 806614 T2947 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 806616 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 806631 T2948 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 806633 T2968 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 806634 T2968 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 806634 T2948 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33852/qwrk/i",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:33852_qwrk%2Fi",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 806636 T2968 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 806646 T2948 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:47892/qwrk/i",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:47892_qwrk%2Fi",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 806646 T2948 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 806647 T2948 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 806659 T2954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 806659 T2967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 806660 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 807637 T2968 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 807638 T2968 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1378855178087/collection1
   [junit4]   2> 807638 T2968 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 807642 T2968 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 807644 T2968 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 807652 T2968 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1378855178087/collection1/'
   [junit4]   2> 807655 T2968 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1378855178087/collection1/lib/classes/' to classloader
   [junit4]   2> 807657 T2968 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1378855178087/collection1/lib/README' to classloader
   [junit4]   2> 807723 T2968 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
   [junit4]   2> 807764 T2968 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 807770 T2968 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 807778 T2968 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 808220 T2968 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 808221 T2968 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 808221 T2968 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 808227 T2968 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 808230 T2968 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 808282 T2968 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 808290 T2968 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 808297 T2968 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 808301 T2968 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 808302 T2968 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 808302 T2968 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 808306 T2968 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 808307 T2968 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 808307 T2968 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 808308 T2968 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1378855178087/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty1/
   [junit4]   2> 808309 T2968 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@57da57da
   [junit4]   2> 808310 T2968 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty1
   [junit4]   2> 808310 T2968 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty1/index/
   [junit4]   2> 808311 T2968 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 808311 T2968 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty1/index
   [junit4]   2> 808312 T2968 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=49.9873046875, floorSegmentMB=1.6240234375, forceMergeDeletesPctAllowed=9.768604114962432, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8977219860458391
   [junit4]   2> 808313 T2968 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a182a18 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ba01ba0),segFN=segments_1,generation=1}
   [junit4]   2> 808313 T2968 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 808321 T2968 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 808322 T2968 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 808323 T2968 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 808323 T2968 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 808324 T2968 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 808324 T2968 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 808325 T2968 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 808326 T2968 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 808326 T2968 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 808327 T2968 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 808328 T2968 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 808329 T2968 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 808329 T2968 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 808330 T2968 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 808331 T2968 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 808332 T2968 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 808336 T2968 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 808336 T2968 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 808337 T2968 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=36.4296875, floorSegmentMB=1.47265625, forceMergeDeletesPctAllowed=14.476735145232706, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 808338 T2968 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a182a18 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ba01ba0),segFN=segments_1,generation=1}
   [junit4]   2> 808339 T2968 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 808339 T2968 oass.SolrIndexSearcher.<init> Opening Searcher@41c141c1 main
   [junit4]   2> 808345 T2969 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@41c141c1 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 808349 T2968 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 808350 T2968 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47892/qwrk/i collection:collection1 shard:shard1
   [junit4]   2> 808350 T2968 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 808366 T2968 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 808398 T2968 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 808405 T2968 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 808406 T2968 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 808406 T2968 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47892/qwrk/i/collection1/
   [junit4]   2> 808407 T2968 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 808408 T2968 oasc.SyncStrategy.syncToMe http://127.0.0.1:47892/qwrk/i/collection1/ has no replicas
   [junit4]   2> 808409 T2968 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47892/qwrk/i/collection1/ shard1
   [junit4]   2> 808410 T2968 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 809689 T2948 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 809712 T2954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 809713 T2967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 809713 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 809746 T2968 oasc.ZkController.register We are http://127.0.0.1:47892/qwrk/i/collection1/ and leader is http://127.0.0.1:47892/qwrk/i/collection1/
   [junit4]   2> 809746 T2968 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47892/qwrk/i
   [junit4]   2> 809747 T2968 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 809747 T2968 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 809748 T2968 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 809752 T2968 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 809756 T2925 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 809757 T2925 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 809758 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 809943 T2925 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 809945 T2925 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44691
   [junit4]   2> 809946 T2925 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 809946 T2925 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 809947 T2925 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1378855182635
   [junit4]   2> 809948 T2925 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1378855182635/'
   [junit4]   2> 809960 T2925 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1378855182635/solr.xml
   [junit4]   2> 810028 T2925 oasc.CoreContainer.<init> New CoreContainer 1437423021
   [junit4]   2> 810029 T2925 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1378855182635/]
   [junit4]   2> 810031 T2925 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 810032 T2925 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 810033 T2925 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 810034 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 810035 T2925 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 810036 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 810037 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 810039 T2925 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 810040 T2925 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 810041 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 810050 T2925 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 810051 T2925 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 810054 T2925 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56132/solr
   [junit4]   2> 810055 T2925 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 810056 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 810059 T2981 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5afb5afb name:ZooKeeperConnection Watcher:127.0.0.1:56132 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 810059 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 810065 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 810073 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 810074 T2983 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73ab73ab name:ZooKeeperConnection Watcher:127.0.0.1:56132/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 810075 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 810093 T2925 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 811110 T2925 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44691_qwrk%2Fi
   [junit4]   2> 811113 T2925 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44691_qwrk%2Fi
   [junit4]   2> 811124 T2954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 811124 T2967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 811139 T2947 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 811139 T2983 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 811140 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 811149 T2954 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 811149 T2967 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 811162 T2984 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 811163 T2984 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 811166 T2984 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 811261 T2948 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 811264 T2948 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:47892/qwrk/i",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:47892_qwrk%2Fi",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 811275 T2948 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44691/qwrk/i",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "node_name":"127.0.0.1:44691_qwrk%2Fi",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 811276 T2948 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 811276 T2948 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 811289 T2954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 811289 T2983 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 811289 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 811289 T2967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 812167 T2984 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 812167 T2984 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1378855182635/collection1
   [junit4]   2> 812168 T2984 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 812171 T2984 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 812171 T2984 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 812176 T2984 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1378855182635/collection1/'
   [junit4]   2> 812178 T2984 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1378855182635/collection1/lib/classes/' to classloader
   [junit4]   2> 812178 T2984 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1378855182635/collection1/lib/README' to classloader
   [junit4]   2> 812230 T2984 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
   [junit4]   2> 812277 T2984 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 812282 T2984 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 812302 T2984 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 812704 T2984 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 812704 T2984 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 812705 T2984 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 812711 T2984 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 812714 T2984 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 812762 T2984 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 812769 T2984 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 812776 T2984 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 812780 T2984 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 812781 T2984 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 812781 T2984 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 812785 T2984 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 812786 T2984 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 812787 T2984 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 812787 T2984 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1378855182635/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty2/
   [junit4]   2> 812788 T2984 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@57da57da
   [junit4]   2> 812789 T2984 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty2
   [junit4]   2> 812790 T2984 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty2/index/
   [junit4]   2> 812790 T2984 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 812791 T2984 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty2/index
   [junit4]   2> 812791 T2984 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=49.9873046875, floorSegmentMB=1.6240234375, forceMergeDeletesPctAllowed=9.768604114962432, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8977219860458391
   [junit4]   2> 812792 T2984 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55f255f2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5fbb5fbb),segFN=segments_1,generation=1}
   [junit4]   2> 812793 T2984 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 812801 T2984 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 812802 T2984 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 812802 T2984 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 812803 T2984 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 812803 T2984 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 812804 T2984 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 812804 T2984 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 812805 T2984 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 812806 T2984 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 812807 T2984 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 812807 T2984 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 812808 T2984 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 812809 T2984 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 812810 T2984 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 812811 T2984 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 812812 T2984 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 812815 T2984 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 812816 T2984 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 812817 T2984 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=36.4296875, floorSegmentMB=1.47265625, forceMergeDeletesPctAllowed=14.476735145232706, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 812818 T2984 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55f255f2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5fbb5fbb),segFN=segments_1,generation=1}
   [junit4]   2> 812819 T2984 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 812819 T2984 oass.SolrIndexSearcher.<init> Opening Searcher@729e729e main
   [junit4]   2> 812825 T2985 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@729e729e main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 812829 T2984 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 812830 T2984 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44691/qwrk/i collection:collection1 shard:shard2
   [junit4]   2> 812830 T2984 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 812838 T2984 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 812867 T2984 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 812874 T2984 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 812875 T2984 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 812875 T2984 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44691/qwrk/i/collection1/
   [junit4]   2> 812876 T2984 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 812876 T2984 oasc.SyncStrategy.syncToMe http://127.0.0.1:44691/qwrk/i/collection1/ has no replicas
   [junit4]   2> 812877 T2984 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44691/qwrk/i/collection1/ shard2
   [junit4]   2> 812877 T2984 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 814327 T2948 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 814380 T2954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 814381 T2967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 814381 T2983 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 814395 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 814398 T2984 oasc.ZkController.register We are http://127.0.0.1:44691/qwrk/i/collection1/ and leader is http://127.0.0.1:44691/qwrk/i/collection1/
   [junit4]   2> 814399 T2984 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44691/qwrk/i
   [junit4]   2> 814400 T2984 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 814400 T2984 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 814401 T2984 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 814470 T2984 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 814476 T2925 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 814477 T2925 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 814478 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 814861 T2925 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 814866 T2925 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41926
   [junit4]   2> 814867 T2925 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 814868 T2925 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 814870 T2925 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1378855187357
   [junit4]   2> 814871 T2925 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1378855187357/'
   [junit4]   2> 814889 T2925 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1378855187357/solr.xml
   [junit4]   2> 814946 T2925 oasc.CoreContainer.<init> New CoreContainer 2026076355
   [junit4]   2> 814947 T2925 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1378855187357/]
   [junit4]   2> 814949 T2925 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 814950 T2925 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 814950 T2925 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 814951 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 814952 T2925 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 814953 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 814953 T2925 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 814954 T2925 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 814955 T2925 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 814956 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 814962 T2925 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 814963 T2925 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 814965 T2925 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56132/solr
   [junit4]   2> 814966 T2925 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 814967 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 814970 T2997 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57155715 name:ZooKeeperConnection Watcher:127.0.0.1:56132 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 814971 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 814992 T2925 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 815003 T2925 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 815029 T2999 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ec82ec8 name:ZooKeeperConnection Watcher:127.0.0.1:56132/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 815030 T2925 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 815049 T2925 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 815988 T2948 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 815992 T2948 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44691/qwrk/i",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "node_name":"127.0.0.1:44691_qwrk%2Fi",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 816016 T2999 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 816016 T2954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 816017 T2983 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 816016 T2967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 816027 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 816068 T2925 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41926_qwrk%2Fi
   [junit4]   2> 816072 T2925 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41926_qwrk%2Fi
   [junit4]   2> 816080 T2954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 816097 T2983 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 816080 T2967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 816099 T2999 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 816103 T2947 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 816104 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 816115 T2954 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 816115 T2983 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 816125 T2967 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 816130 T2999 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 816162 T3000 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 816163 T3000 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 816167 T3000 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 817552 T2948 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 817555 T2948 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:41926/qwrk/i",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core_node_name":"core_node3",
   [junit4]   2> 	  "node_name":"127.0.0.1:41926_qwrk%2Fi",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "core":"collection1"}
   [junit4]   2> 817556 T2948 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 817556 T2948 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 817578 T2954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 817578 T2983 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 817578 T2999 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 817580 T2947 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 817578 T2967 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 818168 T3000 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 818169 T3000 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1378855187357/collection1
   [junit4]   2> 818169 T3000 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 818172 T3000 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 818173 T3000 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 818177 T3000 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1378855187357/collection1/'
   [junit4]   2> 818180 T3000 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1378855187357/collection1/lib/classes/' to classloader
   [junit4]   2> 818181 T3000 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1378855187357/collection1/lib/README' to classloader
   [junit4]   2> 818240 T3000 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
   [junit4]   2> 818320 T3000 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 818325 T3000 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 818332 T3000 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 818749 T3000 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 818750 T3000 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 818751 T3000 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 818756 T3000 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 818760 T3000 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 818812 T3000 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 818819 T3000 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 818827 T3000 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 818831 T3000 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 818832 T3000 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 818833 T3000 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 818839 T3000 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 818840 T3000 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 818841 T3000 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 818841 T3000 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1378855187357/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty3/
   [junit4]   2> 818842 T3000 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@57da57da
   [junit4]   2> 818844 T3000 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty3
   [junit4]   2> 818845 T3000 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty3/index/
   [junit4]   2> 818846 T3000 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty3/index' doesn't exist. Creating new index...
   [junit4]   2> 818847 T3000 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1378855174151/jetty3/index
   [junit4]   2> 818848 T3000 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=49.9873046875, floorSegmentMB=1.6240234375, forceMergeDeletesPctAllowed=9.768604114962432, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8977219860458391
   [junit4]   2> 818849 T3000 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e9e5e9e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36fd36fd),segFN=segments_1,generation=1}
   [junit4]   2> 818849 T3000 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 818857 T3000 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 818858 T3000 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 818858 T3000 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 818859 T3000 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 818860 T3000 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 818860 T3000 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 818861 T3000 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 818861 T3000 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 818862 T3000 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 818863 T3000 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 818864 T3000 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 818864 T3000 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 818865 T3000 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 818866 T3000 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 818867 T3000 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 818868 T3000 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 818872 T3000 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 818872 T3000 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 818873 T3000 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=36.4296875, floorSegmentMB=1.47265625, forceMergeDeletesPctAllowed=14.476735145232706, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 818874 T3000 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e9e5e9e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36fd36fd),segFN=segments_1,generation=1}
   [junit4]   2> 818875 T3000 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 818875 T3000 oass.SolrIndexSearcher.<init> Opening Searcher@410d410d main
   [junit4]   2> 818881 T3001 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@410d410d main{StandardDire

[...truncated too long message...]

DATA:
   [junit4]   1>          {
   [junit4]   1>            "base_url":"http://127.0.0.1:47892/qwrk/i",
   [junit4]   1>            "node_name":"127.0.0.1:47892_qwrk%2Fi",
   [junit4]   1>            "core":"onenodecollectioncore"}
   [junit4]   1>   /solr/aliases.json (0)
   [junit4]   1>   /solr/live_nodes (5)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:48656_qwrk%2Fi (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:41926_qwrk%2Fi (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:44691_qwrk%2Fi (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:47892_qwrk%2Fi (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:33852_qwrk%2Fi (0)
   [junit4]   1>   /solr/clusterstate.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {
   [junit4]   1>         "control_collection":{
   [junit4]   1>           "shards":{"shard1":{
   [junit4]   1>               "range":"80000000-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{"core_node1":{
   [junit4]   1>                   "base_url":"http://127.0.0.1:33852/qwrk/i",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "node_name":"127.0.0.1:33852_qwrk%2Fi",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "leader":"true"}}}},
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "router":"compositeId",
   [junit4]   1>           "replicationFactor":"1"},
   [junit4]   1>         "onenodecollection":{
   [junit4]   1>           "shards":{"shard1":{
   [junit4]   1>               "range":"80000000-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{"core_node1":{
   [junit4]   1>                   "base_url":"http://127.0.0.1:47892/qwrk/i",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "node_name":"127.0.0.1:47892_qwrk%2Fi",
   [junit4]   1>                   "roles":"none",
   [junit4]   1>                   "core":"onenodecollectioncore",
   [junit4]   1>                   "leader":"true"}}}},
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "router":"compositeId",
   [junit4]   1>           "replicationFactor":"1"},
   [junit4]   1>         "collection1":{
   [junit4]   1>           "shards":{
   [junit4]   1>             "shard1":{
   [junit4]   1>               "range":"80000000-ffffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node1":{
   [junit4]   1>                   "base_url":"http://127.0.0.1:47892/qwrk/i",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "node_name":"127.0.0.1:47892_qwrk%2Fi",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node3":{
   [junit4]   1>                   "base_url":"http://127.0.0.1:41926/qwrk/i",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "node_name":"127.0.0.1:41926_qwrk%2Fi",
   [junit4]   1>                   "core":"collection1"}}},
   [junit4]   1>             "shard2":{
   [junit4]   1>               "range":"0-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node2":{
   [junit4]   1>                   "base_url":"http://127.0.0.1:44691/qwrk/i",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "node_name":"127.0.0.1:44691_qwrk%2Fi",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node4":{
   [junit4]   1>                   "base_url":"http://127.0.0.1:48656/qwrk/i",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "node_name":"127.0.0.1:48656_qwrk%2Fi",
   [junit4]   1>                   "core":"collection1"}}}},
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "router":"compositeId",
   [junit4]   1>           "replicationFactor":"1"}}
   [junit4]   1>  /zookeeper (1)
   [junit4]   1>  DATA:
   [junit4]   1>      
   [junit4]   1> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZk2Test -Dtests.method=testDistribSearch -Dtests.seed=AD1928ECDAB6619D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ml_IN -Dtests.timezone=Antarctica/Macquarie -Dtests.file.encoding=UTF-8
   [junit4] ERROR   35.7s J0 | BasicDistributedZk2Test.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:44691/qwrk/i/onenodecollectioncore returned non ok status:404, message:Can not find: /qwrk/i/onenodecollectioncore/update
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AD1928ECDAB6619D:2CFFA6F4ADE901A1]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
   [junit4]    > 	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:738)
   [junit4]   2> 836969 T2925 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 35735 T2924 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> 837748 T3016 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 837749 T3016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 837750 T3016 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Lucene45: {}, docValues:{}, sim=DefaultSimilarity, locale=ml_IN, timezone=Antarctica/Macquarie
   [junit4]   2> NOTE: Linux 3.2.0-52-generic x86/IBM Corporation 1.6.0 (32-bit)/cpus=8,threads=1,free=38026704,total=70255616
   [junit4]   2> NOTE: All tests run in this JVM: [LegacyHTMLStripCharFilterTest, TestManagedSchema, TestBadConfig, TestSuggestSpellingConverter, BadComponentTest, TestCollationKeyRangeQueries, URLClassifyProcessorTest, QueryResultKeyTest, SearchHandlerTest, TestSchemaNameResource, SpellPossibilityIteratorTest, TestSystemIdResolver, LeaderElectionTest, TestSolrQueryParser, BlockCacheTest, SolrXmlInZkTest, TermVectorComponentTest, TestPostingsSolrHighlighter, LoggingHandlerTest, BadCopyFieldTest, ZkControllerTest, TestFaceting, PeerSyncTest, DateMathParserTest, ExternalFileFieldSortTest, FastVectorHighlighterTest, UpdateParamsTest, TestMergePolicyConfig, SimpleFacetsTest, IndexSchemaTest, ClusterStateTest, TestQueryUtils, TestStressVersions, TestClassNameShortening, TestValueSourceCache, TestImplicitCoreProperties, TestSolrDeletionPolicy1, HdfsLockFactoryTest, TestSchemaSimilarityResource, TestReplicationHandler, JsonLoaderTest, SolrInfoMBeanTest, ClusterStateUpdateTest, DistributedSpellCheckComponentTest, CircularListTest, AutoCommitTest, TestDynamicFieldResource, HdfsBasicDistributedZkTest, TestCharFilters, TestConfig, SyncSliceTest, DocumentBuilderTest, SOLR749Test, OverseerTest, WordBreakSolrSpellCheckerTest, TestFieldTypeCollectionResource, UUIDFieldTest, RegexBoostProcessorTest, BasicDistributedZkTest, QueryParsingTest, TestNRTOpen, SliceStateTest, SuggesterTest, BlockDirectoryTest, DirectUpdateHandlerOptimizeTest, OverseerCollectionProcessorTest, TestLMJelinekMercerSimilarityFactory, TestTrie, ResponseLogComponentTest, TestNoOpRegenerator, RecoveryZkTest, SampleTest, FileBasedSpellCheckerTest, TestExtendedDismaxParser, TestSearchPerf, ParsingFieldUpdateProcessorsTest, TestDocumentBuilder, TestJoin, BasicDistributedZk2Test]
   [junit4] Completed on J0 in 36.52s, 1 test, 1 error <<< FAILURES!

[...truncated 552 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:402: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:395: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:455: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1233: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:876: There were test failures: 326 suites, 1450 tests, 1 error, 34 ignored (6 assumptions)

Total time: 53 minutes 0 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk6 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message