lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "none none" <>
Subject Re: Crash / Recovery Scenario
Date Mon, 08 Jul 2002 16:43:40 GMT
 hi, i do perform the same things as you do, but i do that everytime i got a NullPointerException
when i try to run a search . If this happen i try to reopen the index searcher, if i got an
exception here i sleep for 500 ms then i try again, after 5 times i generate a servlet exception.
Concerning the delete of write.lock and commit.lock, i use a manager,what it does is execute
different kind of operation in blocks, like 100 or 1000.
Each operation can be:
1.Delete documents
2.Add documents
3.Search document/s

A combination of this 3 operation allow me to "update" the index with searches still running.
But there is a problem "versioning", between current cache of documents and current version
of "INDEXED" documents, during update you can search for something that is found in the index
but that has been updated in the cache, so i have a bounch of documents duplicate during that,
and at the end i notify using a RMI callback all the clients connected to that Manager to
re open the index, then i clean up all this duplicate. At this stage i have still an error
in case the Manager die because i have all in memory, but i did a little work around to handle
that. My next step is make this "transaction" persistent, so i can recovery the previous "status".

Every time i run an operation as listed above i do a check if "write.lock" or "commit.lock"
exists, in that case i call the unlock() method, i delete them (if the method unlock doesn't),
then i optimize the index.

Until now everything seems to work fine.


On Mon, 8 Jul 2002 09:40:10   
 Nader S. Henein wrote:
>I'm currently using Lucene to sift through about a million documents, I've
>written a servlet to do the indexing and the searching, the servlets are ran
>through resin, The Crash scenario I'm thinking of is a web server crash (
>for a million possible reasons ) while the index is being updated or
>optimized, what I've noticed is the creation of write.lock and commit.lock
>files witch stop further indexing because the application thinks that the
>previously scheduled indexer is still running (witch could very well be true
>depending on the size of the update). This is the recovery I have in mind
>but I think it might be somewhat of a hack, On restart of the web server
>I've written an Init function that checks for write.lock or commit.lock ,
>and if either exist it deletes both of them and optimizes the index. Am I
>forgetting anything ? is this wrong ? is there a Lucene specific way of
>doing this like running the optimizer with a specific setup.
>Nader S. Henein
> , Dubai Internet City
>Tel. +9714 3911900
>Fax. +9714 3911915
>GSM. +9715 05659557
>To unsubscribe, e-mail:   <>
>For additional commands, e-mail: <>

Supercharge your e-mail with a 25MB Inbox, POP3 Access, No Ads
and NoTaglines --> LYCOS MAIL PLUS. 

To unsubscribe, e-mail:   <>
For additional commands, e-mail: <>

View raw message