lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ankit Jain (JIRA)" <>
Subject [jira] [Commented] (LUCENE-8671) Add setting for moving FST offheap/onheap
Date Sat, 09 Feb 2019 03:17:00 GMT


Ankit Jain commented on LUCENE-8671:

Hi David,

Thanks for the feedback.
{quote}Modifying FieldInfo feels wrong to me.  This is a setting that could only apply to
a subset of our PostingsFormat implementations.  It's not fundamental to the metadata FieldInfo
tracks.  I'd prefer a more general per-field name=value setting approach{quote}
I have added more generic reader settings map to FieldInfo in [^offheap_generic_settings.patch]
that can be used for other purposes as well.

{quote}There are plenty of other settings to our postings formats that don't get such 1st
class treatment. It's true that it's not "easy" to make these low-level settings changes but
this doesn't feel like the right way. {quote}
Just for my understanding, since I'm pretty new, can you give example of some of those settings?


> Add setting for moving FST offheap/onheap
> -----------------------------------------
>                 Key: LUCENE-8671
>                 URL:
>             Project: Lucene - Core
>          Issue Type: New Feature
>          Components: core/FSTs, core/store
>            Reporter: Ankit Jain
>            Priority: Minor
>         Attachments: offheap_generic_settings.patch, offheap_settings.patch
>   Original Estimate: 24h
>  Remaining Estimate: 24h
> While LUCENE-8635, adds support for loading FST offheap using mmap, users do not have
the  flexibility to specify fields for which FST needs to be offheap. This allows users to
tune heap usage as per their workload.
> Ideal way will be to add an attribute to FieldInfo, where we have put/getAttribute. Then
FieldReader can inspect the FieldInfo and pass the appropriate On/OffHeapStore when creating
its FST. It can support special keywords like ALL/NONE.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message