lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman SlavĂ­k <slaav...@gmail.com>
Subject Solrj 4.7.2 - slowing down over time
Date Wed, 18 May 2016 14:39:53 GMT
Hi all,

we're using solr in our application and have problem that both searching 
and indexing is slowing down over time.

Versions:
---------
Java 1.7
Solr 4.7.2
Lucene 4.1 (luceneMatchVersion param in solrconfig.xml)

App architecture:
-----------------
We don't use solr as standalone application, but we integrated him with 
solrj library into our app.
It has 1 CoreContainer with 3 cores (let's call them alpha, beta, 
gamma). EmbeddedSolrServer is used as java class to work with each core.
Application does both searching and indexing.

What's going on:
----------------
After Tomcat restart everything is working great for 2-3 days. But after 
this time solr (all cores) starts sloooow down until it's unusable and 
we have to restart Tomcat. Then it works fine.
For example search time for really complex query is 1,5 s when it works 
fine. But then it rises to more than 1 min. Same issue is with indexing 
- first fast but then slow.

Searching:
----------
Core apha is used mainly for normal search. But sometimes for faceting too.
Beta and gamma are only for facets.
alpha: 25000 queries/day
beta: 7000 queries/day
gamma: 7000 queries/day
We do lots of query joins, sometimes cross cores.

Indexing:
---------
We commit changes continuously over day. Number of commits is limited to 
1 commit/min for all three cores. So we do max 1440 commits daily. One 
commit contains between 1 and 100 docs.
Method EmbeddedSolrServer.add(SolrInputDocument) is used and in the end 
EmbeddedSolrServer.commit().
Every night we call EmbeddedSolrServer.optimize() on each core.

Index size:
-----------
alpha: 13,5 GB
beta: 300 MB
gamma: 600 MB

Hardware:
---------
Ubuntu 14.04
8 core CPU
java heap space 22 GB RAM
SSD drive with more than 50 GB free space

Solr config (Same configuration is used for all cores):
-------------------------------------------------------
<directoryFactory name="DirectoryFactory" 
class="${solr.directoryFactory:solr.MMapDirectoryFactory}"/>
<luceneMatchVersion>LUCENE_41</luceneMatchVersion>
<indexConfig>
     <useCompoundFile>false</useCompoundFile>
     <mergeFactor>10</mergeFactor>

     <ramBufferSizeMB>32</ramBufferSizeMB>
     <maxFieldLength>10000</maxFieldLength>
     <writeLockTimeout>1000</writeLockTimeout>
     <commitLockTimeout>10000</commitLockTimeout>

     <lockType>native</lockType>
     <unlockOnStartup>false</unlockOnStartup>
     <reopenReaders>true</reopenReaders>

     <deletionPolicy class="solr.SolrDeletionPolicy">
       <str name="maxCommitsToKeep">1</str>
       <str name="maxOptimizedCommitsToKeep">0</str>
     </deletionPolicy>
</indexConfig>

<updateHandler class="solr.DirectUpdateHandler2" />

<filterCache class="solr.FastLRUCache" size="512" initialSize="512" 
autowarmCount="0"/>
<queryResultCache class="solr.LRUCache" size="512" initialSize="512" 
autowarmCount="0"/>
<documentCache class="solr.LRUCache" size="512" initialSize="512" 
autowarmCount="0"/>

<enableLazyFieldLoading>true</enableLazyFieldLoading>
<useColdSearcher>false</useColdSearcher>
<maxWarmingSearchers>2</maxWarmingSearchers>


Conclusion:
-----------
Is something wrong in configuration? Or is this some kind of bug? Or...?
Can you give me some advice how to resolve this problem please?


Roman











Mime
View raw message