lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ning Li (JIRA)" <>
Subject [jira] Commented: (LUCENE-1335) Correctly handle concurrent calls to addIndexes, optimize, commit
Date Fri, 22 Aug 2008 23:16:44 GMT


Ning Li commented on LUCENE-1335:

I agree that we should not make any API promises about what
it means when the methods (commit, close, rollback, optimize,
addIndexes) are called concurrently from different threads.
The discussion below is on their current behavior.

> Only one addIndexes can run at once, so call to 2nd or more
> addIndexes just blocks until the one is done.

This is achieved by the read-write lock.

> close() and rollback() wait for any running addIndexes to finish
> and then blocks new addIndexes calls

Just to clarify: close(waitForMerges=false) and rollback() make
an ongoing addIndexes[NoOptimize](dirs) abort, but wait for
addIndexes(readers) to finish. It'd be nice if they make any
addIndexes* abort for a quick shutdown, but that's for later.

> commit() waits for any running addIndexes, or any already running
> commit, to finish, then quickly takes a snapshot of the segments
> and syncs the files referenced by that snapshot. While syncing is
> happening addIndexes are then allowed to run again.

commit() and commit(long) use the read-write lock to wait for
a running addIndexes. "committing" is used to serialize commit()
calls. Why isn't it also used to serialize commit(long) calls?

> optimize() is allowed to run concurrently with addIndexes; the two
> simply wait for their respective merges to finish.

This is nice.

More detailed comments:
- In finishMerges, acquireRead and releaseRead are both called.
  Isn't addIndexes allowed again?

- In copyExternalSegments, merges involving external segments
  are carried out in foreground. So why the changes? To relax
  that assumption? But other part still makes the assumption.

- addIndexes(readers) should optimize before startTransaction, no?

- The newly added method segString(dir) in SegmentInfos is
  not used anywhere.

> Correctly handle concurrent calls to addIndexes, optimize, commit
> -----------------------------------------------------------------
>                 Key: LUCENE-1335
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.3, 2.3.1
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.4
>         Attachments: LUCENE-1335.patch, LUCENE-1335.patch, LUCENE-1335.patch
> Spinoff from here:

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message