lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Jelsma <markus.jel...@openindex.io>
Subject RE: Atomic updates to increase single field bulk updates?
Date Wed, 15 Feb 2017 21:39:01 GMT
Hello Sebastian,

Except for the requirement to have all fields stored, there is from Solr/Lucene's point of
view not much difference between indexing a partial update or a complete document. Under the
hood a partial update is a complete object anyway. Using partial updates you gain a little
bandwidth at the expense of additional stored fields.

If your backend is the bottleneck, it would probably be very beneficial for you to switch
to atomic updates: decrease stress on your database and decrease reindexing time.

Regards,
Markus

-----Original message-----
> From:Sebastian Riemer <s.riemer@littera.eu>
> Sent: Wednesday 15th February 2017 19:31
> To: solr-user@lucene.apache.org
> Subject: Atomic updates to increase single field bulk updates?
> 
> Dear solr users,
> 
> when updating documents in bulk (i.e. 40.000 documents at once), and only changing the
value of a single Boolean-Flag, I currently re-index all whole 40.000 objects. However, the
process of obtaining all relevant information for each object from the database is one of
relatively high cost.
> 
> I now wonder, if in this situation it would be a good idea to implement a single-field
update routine using atomic updates? In that case, I could skip any necessary lookups in the
relational database, since the only information would be the new value for that Boolean-Flag,
and the list of those 40.000 document ids.
> 
> I am aware of the requirements to use atomic updates, but as I understood, those would
not have a big impact on performance and only a slight increase in index size?
> 
> What is your opinion on that?
> 
> Thanks for your input, have a nice evening!
> 
> Sebastian
> 
> 

Mime
View raw message