lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jihwan Kim <jihwa...@gmail.com>
Subject Re: Enhancement on a getFloats method of FileFloatSource.java ?
Date Wed, 05 Oct 2016 03:23:30 GMT
Hi Yonik,
I thought about your comment and I might understand what you were saying.
The for loop in the getFloats method assign a different index of the array
whenever a new segment is created/updated. You are saying that is why we
cannot cache the float array as I suggested.  Am I understood correctly?

Thanks.


On Tue, Oct 4, 2016 at 8:59 PM, Jihwan Kim <jihwan11@gmail.com> wrote:

> "The array is indexed by internal lucene docid," --> If I understood
> correctly, it is done inside the for loop that I briefly showed.
>
> In the following code I used, the 'vals' points to an array object and the
> latestCache puts the reference of the same array object.  Then, the index
> decision and assigning a value in the array object is happening inside the
> for loop in the getFloats.  So, the latestCache still hold same array
> object after the index decision is made along with value assignment from a
> value in the external file.
>
> During a QueryHandler (If I understood it correctly), I noticed that
> another array per segment is created (with a size 20 when I looked at) and
> this array is used query results & values in the float array by calling the
> public Object get(IndexReader reader, Object key) method.  This get
> method uses the same float array created by the getFloats.
>
> vals = new float[reader.maxDoc()];
>
> latestCache.put(ffs.field.getName(), vals);
>
> Am I missing something? Any feedback will be helpful to understand the
> Solr better.
>
> Thank you,
>
> Jihwan
>
> On Tue, Oct 4, 2016 at 8:35 PM, Yonik Seeley <yseeley@gmail.com> wrote:
>
>> On Tue, Oct 4, 2016 at 10:09 PM, Jihwan Kim <jihwan11@gmail.com> wrote:
>> > I would like to ask the Solr organization an enhancement request.
>> >
>> > The FileFloatSource creates a cache value from an external file when a
>> Core
>> > is reloaded and/or a new searcher is opened.  Nevertheless, the external
>> > files can be changed less frequently.
>> >
>> [...]
>> > When the version was not changed, we can still use the cached array
>> without
>> > creating a new one.
>>
>> The array is indexed by internal lucene docid, which can change across
>> different views of the index (a commit).
>>
>> What we can do is cache per segment though.  When this class was
>> originally written, we didn't have access to individual segments.
>>
>> -Yonik
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message