lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarrod Cuzens (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-140) docs out of order
Date Tue, 10 Jan 2006 07:32:20 GMT
    [ http://issues.apache.org/jira/browse/LUCENE-140?page=comments#action_12362310 ] 

Jarrod Cuzens commented on LUCENE-140:
--------------------------------------

First I think that Lucene is great and it manages to do an incredible job. However, this issue
is also causing us significant problems. We originally had an AOP interceptor that would update
documents in our HTTP threads however when we started to see this issue we were concerned
that it was caused by multiple threads accessing the index at the same time. We put extra
concurrency controls on the updates using the LuceneIndexAccess API that was posted into bugzilla
by another user. This issue still remained after we added the extra concurrency control.

Since then we have abandoned the AOP approach completely and moved the index rebuild to a
scheduled thread which collects things that were modified from the database (using datestamp
versioning) and then proceeds to update their lucene indexes. We hoped this would solve the
problem because only 1 thread in 1 process would be modifying the index at any given time.
Alas, we are still getting docs out of order exceptions. It is difficult for us to reproduce
as it mainly happens in production and we cannot provide a testcase for it (I wish we could!).


I know that this must be a difficult issue because it is difficult to reproduce. I can't think
of anything strange that we are doing with the indexes (one process, one thread modifying).
This probably isn't much help but I just wanted to let you know that we are also experiencing
the problem. 

> docs out of order
> -----------------
>
>          Key: LUCENE-140
>          URL: http://issues.apache.org/jira/browse/LUCENE-140
>      Project: Lucene - Java
>         Type: Bug
>   Components: Index
>     Versions: unspecified
>  Environment: Operating System: Linux
> Platform: PC
>     Reporter: legez
>     Assignee: Lucene Developers
>  Attachments: bug23650.txt
>
> Hello,
>   I can not find out, why (and what) it is happening all the time. I got an
> exception:
> java.lang.IllegalStateException: docs out of order
>         at
> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:219)
>         at
> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:191)
>         at
> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:172)
>         at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:135)
>         at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:88)
>         at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:341)
>         at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:250)
>         at Optimize.main(Optimize.java:29)
> It happens either in 1.2 and 1.3rc1 (anyway what happened to it? I can not find
> it neither in download nor in version list in this form). Everything seems OK. I
> can search through index, but I can not optimize it. Even worse after this
> exception every time I add new documents and close IndexWriter new segments is
> created! I think it has all documents added before, because of its size.
> My index is quite big: 500.000 docs, about 5gb of index directory.
> It is _repeatable_. I drop index, reindex everything. Afterwards I add a few
> docs, try to optimize and receive above exception.
> My documents' structure is:
>   static Document indexIt(String id_strony, Reader reader, String data_wydania,
> String id_wydania, String id_gazety, String data_wstawienia)
> {
>     Document doc = new Document();
>     doc.add(Field.Keyword("id", id_strony ));
>     doc.add(Field.Keyword("data_wydania", data_wydania));
>     doc.add(Field.Keyword("id_wydania", id_wydania));
>     doc.add(Field.Text("id_gazety", id_gazety));
>     doc.add(Field.Keyword("data_wstawienia", data_wstawienia));
>     doc.add(Field.Text("tresc", reader));
>     return doc;
> }
> Sincerely,
> legez

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message