lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Modassar Ather <modather1...@gmail.com>
Subject Re: Very high memory and CPU utilization.
Date Mon, 02 Nov 2015 12:21:52 GMT
I monitored swap activities for the query using vmstat. The *so* and *si*
shows 0 till the completion of query. Also the top showed 0 against swap.
This means there was no scarcity of physical memory. Swap activity seems
not to be a bottleneck.
Kindly note that this I ran on 8 node cluster with 30 gb RAM and 140 gb of
index on each node.

Regards,
Modassar

On Mon, Nov 2, 2015 at 5:27 PM, Modassar Ather <modather1981@gmail.com>
wrote:

> Okay. I guess your observation of 400% for a single core is with top and
> looking at that core's entry? If so, the 400% can be explained by
> excessive garbage collection. You could turn GC-logging on to check
> that. With a bit of luck GC would be the cause of the slow down.
>
> Yes it is with top command. I will check GC activities and try to relate
> with CPU usage.
>
> The query q=network se* is quick enough in our system too. It takes around
> 3-4 seconds for around 8 million records.
> The problem is with the same query as phrase. q="network se*".
> Can you please share your experience with such query where the wild card
> expansion is huge like in the query above?
>
> I changed my SolrCloud setup from 12 shard to 8 shard and given each shard
> 30 GB of RAM on the same machine with same index size (re-indexed) but
> could not see the significant improvement for the query given.
>
> I will check the swap activity.
>
> Also can you please share your experiences with respect to RAM, GC, solr
> cache setup etc as it seems by your comment that the SolrCloud environment
> you have is kind of similar to the one I work on?
>
> Regards,
> Modassar
>
> On Mon, Nov 2, 2015 at 5:20 PM, Toke Eskildsen <te@statsbiblioteket.dk>
> wrote:
>
>> On Mon, 2015-11-02 at 16:25 +0530, Modassar Ather wrote:
>> > The remaining size after you removed the heap usage should be reserved
>> for
>> > the index (not only the other system activities).
>> > I am not able to get  the above point. So when I start Solr with 28g
>> RAM,
>> > for all the activities related to Solr it should not go beyond 28g. And
>> the
>> > remaining heap will be used for activities other than Solr. Please help
>> me
>> > understand.
>>
>> It is described here:
>> https://wiki.apache.org/solr/SolrPerformanceProblems#OS_Disk_Cache
>>
>> I will be quick to add that I do not agree with Shawn (the primary
>> author of the page) on the stated limits and find that the page in
>> general ignores that performance requirements differ a great deal.
>> Nevertheless, it is very true that Solr performance is tied to the
>> amount of OS disk cache:
>>
>> You can have a machine with 10TB of RAM, but Solr performance will still
>> be poor if you use it all for JVMs.
>>
>> Practically all modern operating system uses free memory for disk cache.
>> Free memory is the memory not used for JVMs or other programs. It might
>> be that you have a lot less than 30-40GB free: If you are on a Linux
>> server, try calling 'top' and see what is says under 'cached'.
>>
>> Related, I support jim's suggestion to inspect the swap activity:
>> In the past we had problem with a machine that insisted on swapping
>> excessively, although there were high IO and free memory.
>>
>> > The disks are SSDs.
>>
>> That makes your observations stranger still.
>>
>>
>> - Toke Eskildsen, State and University Library, Denmark
>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message