lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: SolrCloud large transaction logs
Date Thu, 10 Jan 2013 17:01:36 GMT
I think it really depends - if you are gong for very fast visibility, your going to spend a
bunch of time warming, and then just throw it out before it even gets much if any reuse. For
very fast visibility turnaround, I suspect you don't want to do any warming. I think it depends
on many things.

One of the tradeoffs off using a very fast soft commit is that Sol's std caches will not be
nearly as useful.

- Mark

On Jan 10, 2013, at 11:24 AM, Upayavira <> wrote:

> That's great Mark. Thx. One final question... all the stuff to do with
> autowarming and static warming of caches - I presume all of that
> configuration is still relevant (if less so) as you still need to warm
> caches on a soft commit, even if those caches are much smaller than they
> would be otherwise?
> Thanks! Upayavira
> On Thu, Jan 10, 2013, at 04:18 PM, Mark Miller wrote:
>> There is no need to open a Searcher because you are controlling
>> visibility through the faster 'soft' commit. That will reopen the reader
>> from the IndexWriter. Because of that, there is no reason to do a heavy,
>> non NRT Searcher reopen on hard commits. Essentially, the hard commit
>> becomes simply about periodically flushing the tlog and the soft commit
>> completely controls visibility.
>> - Mark
>> On Jan 10, 2013, at 9:41 AM, Upayavira <> wrote:
>>> And you don't need to open a searcher (openSearcher=false) because
>>> you've got caches built up already alongside the in-memory NRT segment
>>> which you can continue to use once the hard commit has happened? Is that
>>> correct?
>>> (sorry for hijacking the thread - hopefully it is somewhat relevant)
>>> Upayavira
>>> On Thu, Jan 10, 2013, at 02:18 PM, Mark Miller wrote:
>>>> Setup hard auto commit with openSeacher=false. I would do it at least
>>>> once a minute. Don't worry about the commit being out of sync on the
>>>> different nodes - you will be using soft commits for visibility. The hard
>>>> commits will just be about relieving the pressure on the tlog.
>>>> - Mark
>>>> On Jan 10, 2013, at 6:43 AM, gadde <> wrote:
>>>>> we have a SolrCloud with 3 nodes. we add documents to leader node and
>>>>> commitwithin(100secs) option in SolrJ to add documents. AutoSoftCommit
>>>>> SolrConfig is 1000ms.
>>>>> Transaction logs on replicas grew bigger than the index and we ran out
>>>>> disk space in few days. Leader's tlogs are very small in few hundred
>>>>> The following post suggest hard commit is required for "relieving the
>>>>> pressure of the transactionlog"
>>>>> what is the best way to do a hard commit on this setup in SolrCloud?
>>>>> a. Through autoCommit in SolrConfig? which would cause hard commit on
>>>>> the nodes at different times b. Trigger hard commit on leader while updating
>>>>> through SolrJ?
>>>>> Thanks
>>>>> Shyam
>>>>> --
>>>>> View this message in context:
>>>>> Sent from the Solr - User mailing list archive at

View raw message