Heh, the "it depends" answer :-)
Thanks for the clarification.
Upayavira
On Thu, Jan 10, 2013, at 05:01 PM, Mark Miller wrote:
> I think it really depends - if you are gong for very fast visibility,
> your going to spend a bunch of time warming, and then just throw it out
> before it even gets much if any reuse. For very fast visibility
> turnaround, I suspect you don't want to do any warming. I think it
> depends on many things.
>
> One of the tradeoffs off using a very fast soft commit is that Sol's std
> caches will not be nearly as useful.
>
> - Mark
>
> On Jan 10, 2013, at 11:24 AM, Upayavira <uv@odoko.co.uk> wrote:
>
> > That's great Mark. Thx. One final question... all the stuff to do with
> > autowarming and static warming of caches - I presume all of that
> > configuration is still relevant (if less so) as you still need to warm
> > caches on a soft commit, even if those caches are much smaller than they
> > would be otherwise?
> >
> > Thanks! Upayavira
> >
> > On Thu, Jan 10, 2013, at 04:18 PM, Mark Miller wrote:
> >> There is no need to open a Searcher because you are controlling
> >> visibility through the faster 'soft' commit. That will reopen the reader
> >> from the IndexWriter. Because of that, there is no reason to do a heavy,
> >> non NRT Searcher reopen on hard commits. Essentially, the hard commit
> >> becomes simply about periodically flushing the tlog and the soft commit
> >> completely controls visibility.
> >>
> >> - Mark
> >>
> >> On Jan 10, 2013, at 9:41 AM, Upayavira <uv@odoko.co.uk> wrote:
> >>
> >>> And you don't need to open a searcher (openSearcher=false) because
> >>> you've got caches built up already alongside the in-memory NRT segment
> >>> which you can continue to use once the hard commit has happened? Is that
> >>> correct?
> >>>
> >>> (sorry for hijacking the thread - hopefully it is somewhat relevant)
> >>>
> >>> Upayavira
> >>>
> >>> On Thu, Jan 10, 2013, at 02:18 PM, Mark Miller wrote:
> >>>> Setup hard auto commit with openSeacher=false. I would do it at least
> >>>> once a minute. Don't worry about the commit being out of sync on the
> >>>> different nodes - you will be using soft commits for visibility. The
hard
> >>>> commits will just be about relieving the pressure on the tlog.
> >>>>
> >>>> - Mark
> >>>>
> >>>> On Jan 10, 2013, at 6:43 AM, gadde <gadde.sai@gmail.com> wrote:
> >>>>
> >>>>> we have a SolrCloud with 3 nodes. we add documents to leader node
and use
> >>>>> commitwithin(100secs) option in SolrJ to add documents. AutoSoftCommit
in
> >>>>> SolrConfig is 1000ms.
> >>>>>
> >>>>> Transaction logs on replicas grew bigger than the index and we ran
out of
> >>>>> disk space in few days. Leader's tlogs are very small in few hundred
MBs.
> >>>>>
> >>>>> The following post suggest hard commit is required for "relieving
the memory
> >>>>> pressure of the transactionlog"
> >>>>> http://lucene.472066.n3.nabble.com/SolrCloud-is-softcommit-cluster-wide-for-the-collection-td4021584.html#a4021631
> >>>>>
> >>>>> what is the best way to do a hard commit on this setup in SolrCloud?
> >>>>>
> >>>>> a. Through autoCommit in SolrConfig? which would cause hard commit
on all
> >>>>> the nodes at different times b. Trigger hard commit on leader while
updating
> >>>>> through SolrJ?
> >>>>>
> >>>>> Thanks
> >>>>> Shyam
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-large-transaction-logs-tp4032160.html
> >>>>> Sent from the Solr - User mailing list archive at Nabble.com.
> >>>>
> >>
>
|