lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Bernstein <joels...@gmail.com>
Subject Re: Solrj 4.7.2 - slowing down over time
Date Wed, 18 May 2016 21:24:19 GMT
One thing to investigate is whether your caches are too large and gradually
filling up memory. It does sound like memory is getting tighter over time.
A memory profiler would be helpful in figuring out memory issues.

Moving to autoCommits would also eliminate any slowness due to overlapping
searchers from committing too frequently.



Joel Bernstein
http://joelsolr.blogspot.com/

On Wed, May 18, 2016 at 10:39 AM, Roman SlavĂ­k <slaava87@gmail.com> wrote:

> Hi all,
>
> we're using solr in our application and have problem that both searching
> and indexing is slowing down over time.
>
> Versions:
> ---------
> Java 1.7
> Solr 4.7.2
> Lucene 4.1 (luceneMatchVersion param in solrconfig.xml)
>
> App architecture:
> -----------------
> We don't use solr as standalone application, but we integrated him with
> solrj library into our app.
> It has 1 CoreContainer with 3 cores (let's call them alpha, beta, gamma).
> EmbeddedSolrServer is used as java class to work with each core.
> Application does both searching and indexing.
>
> What's going on:
> ----------------
> After Tomcat restart everything is working great for 2-3 days. But after
> this time solr (all cores) starts sloooow down until it's unusable and we
> have to restart Tomcat. Then it works fine.
> For example search time for really complex query is 1,5 s when it works
> fine. But then it rises to more than 1 min. Same issue is with indexing -
> first fast but then slow.
>
> Searching:
> ----------
> Core apha is used mainly for normal search. But sometimes for faceting too.
> Beta and gamma are only for facets.
> alpha: 25000 queries/day
> beta: 7000 queries/day
> gamma: 7000 queries/day
> We do lots of query joins, sometimes cross cores.
>
> Indexing:
> ---------
> We commit changes continuously over day. Number of commits is limited to 1
> commit/min for all three cores. So we do max 1440 commits daily. One commit
> contains between 1 and 100 docs.
> Method EmbeddedSolrServer.add(SolrInputDocument) is used and in the end
> EmbeddedSolrServer.commit().
> Every night we call EmbeddedSolrServer.optimize() on each core.
>
> Index size:
> -----------
> alpha: 13,5 GB
> beta: 300 MB
> gamma: 600 MB
>
> Hardware:
> ---------
> Ubuntu 14.04
> 8 core CPU
> java heap space 22 GB RAM
> SSD drive with more than 50 GB free space
>
> Solr config (Same configuration is used for all cores):
> -------------------------------------------------------
> <directoryFactory name="DirectoryFactory"
> class="${solr.directoryFactory:solr.MMapDirectoryFactory}"/>
> <luceneMatchVersion>LUCENE_41</luceneMatchVersion>
> <indexConfig>
>     <useCompoundFile>false</useCompoundFile>
>     <mergeFactor>10</mergeFactor>
>
>     <ramBufferSizeMB>32</ramBufferSizeMB>
>     <maxFieldLength>10000</maxFieldLength>
>     <writeLockTimeout>1000</writeLockTimeout>
>     <commitLockTimeout>10000</commitLockTimeout>
>
>     <lockType>native</lockType>
>     <unlockOnStartup>false</unlockOnStartup>
>     <reopenReaders>true</reopenReaders>
>
>     <deletionPolicy class="solr.SolrDeletionPolicy">
>       <str name="maxCommitsToKeep">1</str>
>       <str name="maxOptimizedCommitsToKeep">0</str>
>     </deletionPolicy>
> </indexConfig>
>
> <updateHandler class="solr.DirectUpdateHandler2" />
>
> <filterCache class="solr.FastLRUCache" size="512" initialSize="512"
> autowarmCount="0"/>
> <queryResultCache class="solr.LRUCache" size="512" initialSize="512"
> autowarmCount="0"/>
> <documentCache class="solr.LRUCache" size="512" initialSize="512"
> autowarmCount="0"/>
>
> <enableLazyFieldLoading>true</enableLazyFieldLoading>
> <useColdSearcher>false</useColdSearcher>
> <maxWarmingSearchers>2</maxWarmingSearchers>
>
>
> Conclusion:
> -----------
> Is something wrong in configuration? Or is this some kind of bug? Or...?
> Can you give me some advice how to resolve this problem please?
>
>
> Roman
>
>
>
>
>
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message