lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gora Mohanty <g...@mimirtech.com>
Subject Re: Improving Solr performance
Date Fri, 14 Jan 2011 05:05:47 GMT
On Thu, Jan 13, 2011 at 10:10 PM, supersoft <elarablog@gmail.com> wrote:
>
> On the one hand, I found really interesting those comments about the reasons
> for sharding. Documentation agrees you about why to split an index in
> several shards (big sizes problems) but I don't find any explanation about
> the inconvenients as an Access Control List. I guess there should be some
> and they can be critical in this design. Any example?
[...]

Can I ask what might be a stupid question? How are you measuring
the numbers below, and what do they mean?

As your hit ratio is close to 1 (i.e., everything after the first query is
coming from the cache), these numbers seem a little strange. Are
these really the time for each of the N simultaneous queries? They
seem to be monotonically increasing (though with a couple of
strange exceptions), which leads me to suspect that they are some
kind of cumulative times, e.g., by this interpretation, for the case of
the 10 simultaneous queries, the first one takes 1047ms, the second
268ms, the third 125ms, and so on.

We have run performance tests with pg_bench on a index of size
40GB on a single Solr server with about 6GB of RAM allocated
to Solr, and see what I would think of as expected behaviour, i.e.,
for every fresh query term, the first query takes the longest, and
the time for subsequent queries with the same term goes down
dramatically, as the result is coming out of the cache. This is at
odds to what you describe here, so I have to go back and check
that we did not miss something important.

> 1 simultaneous query: 3437 ms (cache fails)
>
> 2 simultaneous queries: 594, 954 ms
>
> 10 simultaneous queries: 1047, 1313, 1438, 1797, 1922, 2094, 2250, 2500,
> 2938, 3000 ms
>
> 50 simultaneous queries: 1203, 1453, 1453, 1437, 1625, 1953, 5688, 12938,
> 14953, 16281, 15984, 16453, 15812, 16469, 16563, 16844, 17703, 16843, 17359,
> 16828, 18235, 18219, 18172, 18203, 17672, 17344, 17453, 18484, 18157, 18531,
> 18297, 18359, 18063, 18516, 18125, 17516, 18562, 18016, 18187, 18610, 18703,
> 18672, 17829, 18344, 18797, 18781, 18265, 18875, 18250, 18812
>
> 100 simultaneous queries: 1297, 1531, 1969, 2203, 2375, 2891, 3937, 4109,
> 4703, 4890, 5047, 5312, 5563, 6422, 6437, 7063, 7093, 7391, 7594, 7672,
> 8172, 8547, 8750, 8984, 9265, 9609, 9907, 10344, 11406, 11484, 11484, 11500,
> 11547, 11703, 11797, 11875, 11922, 12328, 12375, 12875, 12922, 13187, 13219,
> 13407, 13500, 13562, 13719, 13828, 13875, 14016, 14078, 14672, 15922, 16328,
> 16625, 16953, 17282, 18172, 18484, 18985, 20594, 20625, 20860, 21281, 21469,
> 21625, 21875, 21875, 22141, 22157, 22172, 23125, 23125, 23141, 23203, 23203,
> 23328, 24625, 24641, 24672, 24797, 24985, 25031, 25188, 25844, 25937, 26016,
> 26437, 26453, 26437, 26485, 28297, 28687, 31782, 31985, 31969, 32016, 32031,
> 32016, 32281 ms
[...]

Regards,
Gora

Mime
View raw message