lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5310) Merge Threads unnecessarily block on SerialMergeScheduler
Date Sun, 27 Apr 2014 06:56:15 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13982208#comment-13982208
] 

Simon Willnauer commented on LUCENE-5310:
-----------------------------------------

hey mike, thanks for taking a look at this but your patch doesn't implement what I tried to
achieve here and has some problems in SMS. In SMS there is a chance for 2 or more threads
to pick a merge since the critical section between maybeStall() and writer.merge(merge); is
not synced so if 2 threads check if they need to stall they can both make progress and pass
the check before the first one is picked up? There is also no chance to simply return if stalling
is required and notify some other component to handle the merge-falling-behind situation.
You still block the incoming thread no matter what or you continue to pick a merge. I also
really don't like that we rely on #notify & #wait since they can be triggered from outside
of the class and from subclasses. I think in our case we should use a private condition here
instead. yet  if we move to a condition and maybe signal with a boolean returned from maybeStall()
if we should continue merging we can merge the two approaches and make them safe. 

> Merge Threads unnecessarily block on SerialMergeScheduler
> ---------------------------------------------------------
>
>                 Key: LUCENE-5310
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5310
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/index
>    Affects Versions: 4.5, 5.0
>            Reporter: Simon Willnauer
>            Priority: Minor
>             Fix For: 4.9, 5.0
>
>         Attachments: LUCENE-5310.patch, LUCENE-5310.patch, LUCENE-5310.patch, LUCENE-5310.patch,
LUCENE-5310.patch
>
>
> I have been working on a high level merge multiplexer that shares threads across different
IW instances and I came across the fact that SerialMergeScheduler actually blocks incoming
thread is a merge in going on. Yet this blocks threads unnecessarily since we pull the merges
in a loop anyway. We should use a tryLock operation instead of syncing the entire method?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message