lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <>
Subject Re: Benchmarking Solr 3.3 vs. 4.0
Date Thu, 29 Nov 2012 16:40:10 GMT
On 11/29/2012 8:29 AM, Daniel Exner wrote:
> I'll answer both your mails in one.
> Shawn Heisey wrote:
>> On 11/29/2012 3:15 AM, Daniel Exner wrote:
>>> I'm currently doing some benchmarking of a real Solr 3.3 instance vs
>>> the same ported to Solr 4.0.
> [..]
>>> In the graph you can see high CPU load, all the time. This is even the
>>> case if I reduce the QPS down to 5, so CPU is no good metric for
>>> comparison between Solr 3.3 and 4.0 (at least on this machine).
>>> The missing memory data is due to the PerfMon JMeter Plugin having
>>> time-outs sometimes.
>>> You can also see no real increase in latency when pushing data into
>>> the index. This is puzzling me, as rumours say one should not push new
>>> data while under high load, as this would hurt query performance.
>> I don't see any attachments, or any links to external attachments, so I
>> can't see the graph.  I can only make general statements, and I can't
>> guarantee that they'll even be applicable to your scenario.  You may
>> need to use an external attachment service and just send us a link.
> Indeed, it seems like the mailing list daemon scrubbed my attachement. 
> I dropped it into my Dropbox, here
>> Are you seeing lower performance, or just worried about the CPU load?
>> Solr4 should be able to handle concurrent indexing and querying better
>> than 3.x.  It is able to do things concurrently that were not possible
>> before.
> In general I'm interested in how much better Solr 4 performs and if it 
> may be feasonable to use less powerful machines to get the same low 
> latency, or do more data pushes etc.
>> One way that performance improvements happen is that developers find
>> slow sections of code where the CPU is fairly idle, and rewrite them so
>> they are faster, but also exercise the CPU harder. When the new code
>> runs, CPU load goes higher, but it all runs faster.
> Graphs show a slightly better latency for Solr 4.0 compared to 3.3, 
> but not while pushing data.
>> Another note specifically related to this part: Have you used the same
>> configuration and done the minimal changes required to make it run, or
>> have you tried to update the config for 4.0 and its considerable list of
>> new features?  Did you start with a blank index on 4.0, or did you copy
>> the 3.3 index over?
> I used the same configuration and did the minimal changes.
> The first runs where using the same data from Solr 3.3 in Solr 4.0 (in 
> fact it was even the same data dir..) but further runs used freshly 
> filled different indices.

For best results, you'll want to ensure that Solr4 is working completely 
from scratch, that it has never seen a 3.3 index, so that it will use 
its own native format.  It may be a good idea to look into the example 
Solr4 config/schema and see whether there are improvements you can 
make.  One note: the updateLog feature in the update handler config will 
generally cause performance to be lower.  The features that require 
updateLog would make this less of an apples to apples comparison, so I 
wouldn't enable it unless I knew I needed it.

Unless the lines are labelled wrong in the legend, the graph does show 
higher CPU usage during the push, but lower CPU usage during the 
optimize and most of the rest of the time.

The graph shows that Solr4 has lower latency than 3.3 during both the 
push and the optimize, as well as most of the rest of the time.  The 
latency numbers however are a lot higher than I would expect, seeming to 
average out at around 100 seconds (100000 ms).  That is terrible 
performance from both versions.  On my own Solr installation, which is 
distributed and has 78 million documents, I have a median latency of 8 
milliseconds and a 95th percentile latency of 248 milliseconds.

Is this a 64-bit platform with a 64-bit Java?  How much memory have you 
allocated for the java heap?  How big is the index?


View raw message