lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Facet shows deleted values...
Date Tue, 05 Jan 2016 01:15:21 GMT
On 1/4/2016 4:11 PM, Don Bosco Durai wrote:
> Erick, I am using SolrCloud with solrconfig.xml configured with autoCommit. And I also
read somewhere that explicit commit is not recommended in SolrCloud mode. Regarding auto warm,
my server has/was been running for a while.

Since 4.0, autoCommit with openSearcher set to false is highly
recommended, no matter what your needs are regarding visibility, and
whether or not you're running in cloud mode.  The exact interval to use
is a subject for vigorous debate.  A common maxTime value that you will
see for autoCommit is 15 seconds (15000).  I personally feel this is too
frequent, but many people use that value with no problems.  I use five
minutes (300000) in my own config, but over the course of those five
minutes, there's not much in the way of updates, so the log replay will
take very little time.  Using autoCommit with openSearcher set to false
takes care of transaction log rotation, it doesn't do ANYTHING for
document visibility.

The issue of how to handle document visibility will depend on exactly
how you use your index.  Do not worry about whether the index is
SolrCloud or not for this topic.

One way of handling document visibility is to use autoSoftCommit
(available since 4.0) in your config ... with maxTime set to the longest
possible interval you can stand.  My personal recommendation is to never
set that interval shorter than one minute (60000).  Push back if you are
told that documents must be visible faster than that.  If you use
autoSoftCommit, you won't need explicit commits from your indexing
application.

Another way to handle document visibility is the commitWithin parameter
on each update request.  This is similar to autoSoftCommit, but gets set
on the update request.  Just like autoSoftCommit, I would not recommend
a value less than one minute, and if this parameter is used on all
updates, you will never need an explicit commit.

Using autoSoftCommit or commitWithin is a good option if there are many
clients/threads sending changes to the same index or the indexing
happens in bursts where the update size is wildly different and
completely unpredictable.

The final way to handle document visibility is explicit commits.  When
you want changes to be visible, you send a commit, hard or soft, with
openSearcher set to true (this is the default for this parameter), and a
short time later, all changes sent before that commit will become
visible.  This is how I handle my own index.  This is a good option if
all indexing is coming from a single source and that source has complete
control over all indexing operations.

One of the strong goals with commits is to avoid them happening too
frequently, so they don't overlap, and so the machine is spending less
time handling commits than it spends either idle or handling queries.

Here's a blog post with more detail.  The blog post says "SolrCloud" but
almost all of it is equally applicable to Solr 4.x and 5.x indexes that
are not running in cloud mode:

http://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

Thanks,
Shawn


Mime
View raw message