lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emir Arnautović <emir.arnauto...@sematext.com>
Subject Re: Searching for an efficient and scalable way to filter query results using non-indexed and dynamic range values
Date Thu, 01 Feb 2018 10:18:23 GMT
Hi,
I did not check it in code, but based on earlier comments on ML, it seems that in place updates
are not as it sounds - it will rewrite doc values for the segment that is updated. If you
really want to avoid index changes, you can maybe use external field: https://lucene.apache.org/solr/guide/6_6/working-with-external-files-and-processes.html
<https://lucene.apache.org/solr/guide/6_6/working-with-external-files-and-processes.html>
and function queries when evaluating.

HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 1 Feb 2018, at 11:10, Alessandro Benedetti <a.benedetti@sease.io> wrote:
> 
> Reading from the wiki [1]: 
> 
> " An atomic update operation is performed using this approach only when the
> fields to be updated meet these three conditions:
> 
> are non-indexed (indexed="false"), non-stored (stored="false"), single
> valued (multiValued="false") numeric docValues (docValues="true") fields;
> 
> the _version_ field is also a non-indexed, non-stored single valued
> docValues field; and,
> 
> copy targets of updated fields, if any, are also non-indexed, non-stored
> single valued numeric docValues fields. "
> 
> Now, following Diego's idea, if you model your data as a single valued
> incremental numeric field, you should not index it.
> This can be a blocker, but I remember that even if you don't index a field,
> it is possible to search on it using the docValues data structure.
> But I have no idea if the performance are going to be acceptable in your
> case.
> IntPoint seems compatible with docValues [2]:
> 
> "Int|Long|Float|Double|Date PointField
> 
> If the field is single-valued (i.e., multi-valued is false), Lucene will use
> the NUMERIC type."
> 
> I would probably give it a shot.
> But is it compatible with what you need ? Is the incremental crawling cycle
> good enough ? don't you need to associate the Crawling cycle to the
> Experiment Id ?
> 
> Cheers
> 
> 
> 
> [1]
> https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-In-PlaceUpdates
> [2] https://lucene.apache.org/solr/guide/6_6/docvalues.html
> 
> 
> 
> -----
> ---------------
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message