lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: ParalleReader and synchronization between indexes
Date Wed, 30 Apr 2008 21:48:33 GMT
Rajesh,

You are asking a fairly complicated question on a seldom used piece of  
functionality.  Constantly pinging the list is just making it less  
likely that someone will respond with an answer.  The likelihood that  
the 1 person who understand that code (and trust me, it really is  
likely very few people who know how to practically employ it) enough  
to give practical advice have read it in the time period you have  
alloted us to respond is next to nil.   We are all volunteers with day  
jobs.

Have you bothered to search the dev and user mailing list for  
information on the class in question?  I would look for threads from  
Doug or Chuck Williams.

-Grant


On Apr 30, 2008, at 5:00 PM, Rajesh parab wrote:

> Hi Guys,
>
> Any comments on this?
>
> I was looking into Lucene archive and came across this
> thread what asks the same question.
>
> http://www.gossamer-threads.com/lists/lucene/java-user/50477?search_string=parallelreader;#50477
>
> Any pointers will be helpful.
>
> Regards,
> Rajesh
>
> --- Rajesh parab <rajesh_parab_1@yahoo.com> wrote:
>
>> Hi All,
>>
>> Any suggestions/comments on my questions in this
>> thread will be really helpful.
>>
>> We are planning to use Lucene indexes throughout the
>> application and exploring possibilites of
>> partitioning
>> data between multiple indexes.
>>
>> Regards,
>> Rajesh
>>
>> --- Rajesh parab <rajesh_parab_1@yahoo.com> wrote:
>>
>>> Hi,
>>>
>>> This is from javadoc of ParallelReader:
>>>
>>>
>>
> ======================================================
>>>
>>> An IndexReader which reads multiple, parallel
>>> indexes.
>>> Each index added must have the same number of
>>> documents, but typically each contains different
>>> fields. Each document contains the union of the
>>> fields
>>> of all documents with the same document number.
>> When
>>> searching, matches for a query term are from the
>>> first
>>> index added that has the field.
>>>
>>> This is useful, e.g., with collections that have
>>> large
>>> fields which change rarely and small fields that
>>> change more frequently. The smaller fields may be
>>> re-indexed in a new index and both indexes may be
>>> searched together.
>>>
>>>
>>
> ======================================================
>>>
>>> I have a similar use case as mentioned above and
>>> hence
>>> would like to use ParallelReader to search across
>>> multiple indexes.
>>>
>>> I have an object that has 50 fields. Out of these
>> 50
>>> fields, 45 are relatively static and other 5 are
>>> modified very often. So, I am planning to
>> partition
>>> this objects data into 2 indexes such that 45
>> static
>>> fields will be part of one index and remaining 5
>>> dynamic fields will constitute second index. While
>>> generating the index for the first time, I can
>> make
>>> sure that the document order for documents inside
>>> both
>>> these indexes is same and hence ParallelReader
>> will
>>> work properly with it.
>>>
>>> The question is -
>>> What if the data inside second (smaller) index
>>> changes? In order to update index document, I will
>>> have to delete it and re-insert it again as Lucene
>>> does not support document update. This action (of
>>> delete and re-insert) will change internal
>> document
>>> id
>>> for updated document inside second index and in
>>> order
>>> to sync it with first index, I will have to also
>>> modify first (relatively big and static) index. If
>>> we
>>> will have to update both the indexes, how it is
>>> different from having a single index with all the
>>> fields? What is the use case in which
>> ParallelReader
>>> will get used? As per documentation, I was
>> thinking
>>> that it will apply for my use case, but
>>> synchronizing
>>> the indexes seems to be a problem.
>>>
>>> Please help.
>>>
>>> Regards,
>>> Rajesh
>>>
>>>
>>>
>>>
>>>
>>
> ____________________________________________________________________________________
>>> Be a better friend, newshound, and
>>> know-it-all with Yahoo! Mobile.  Try it now.
>>>
>>
> http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
>>>
>>>
>>
> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail:
>>> java-user-help@lucene.apache.org
>>>
>>>
>>
>>
>>
>>
>>
> ____________________________________________________________________________________
>> Be a better friend, newshound, and
>> know-it-all with Yahoo! Mobile.  Try it now.
>>
> http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
>>
>>
> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail:
>> java-user-help@lucene.apache.org
>>
>>
>
>
>
>       
> ____________________________________________________________________________________
> Be a better friend, newshound, and
> know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

--------------------------
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ







---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message