lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bictor Man <bictor...@gmail.com>
Subject drastic performance decrease with 20 cores
Date Mon, 26 Sep 2011 15:33:54 GMT
Hi everyone,

Sorry if this issue has been discussed before, but I'm new to the list.

I have a solr (3.4) instance running with 20 cores (around 4 million docs
each).
The instance has allocated 13GB in a 16GB RAM server. If I run several sets
of queries sequentially in each of the cores, the I/O access goes very high,
so does the system load, while the CPU percentage remains low.
It takes almost 1 hour to complete the set of queries.

If I stop solr and restart it with 6GB allocated and 10 cores, after a bit
the I/O access goes down and the CPU goes up, taking only around 5 minutes
to complete all sets of queries.

Meaning that for me is MUCH more performant having 2 solr instances running
with half the data and half the memory than a single instance will all the
data and memory.

It would be even way faster to have 1 instance with half the cores/memory,
run the queues, shut it down, start a new instance and repeat the process
than having a big instance running everything.

Furthermore, if I take the 20cores/13GB instance, unload 10 of the cores,
trigger the garbage collector and run the sets of queries again, the
behavior still remains slow taking like 30 minutes.

am I missing something here? does solr change its caching policy depending
on the number of cores at startup or something similar?

Any hints will be very appreciated.

Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message