lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject RE: DovValues and in-place udpates
Date Mon, 12 Feb 2018 18:17:42 GMT

: True, I could remove the trigger to rebuild the entire document. But 
: what if a different field changes and the whole document is triggered 
: for update for a different field. We have the same problem.

at a high level, your concern is really compleltey orthoginal to the
question of in-place updates, it's a broader question of having 2 diff    
systems that might want to modify the same document in solr, but one
system is "slower" then the other (because it has to fetch more external  
data or only operates in batches, etc...)

This is where things like optimistic concurrency are really powerful.

When you trigger your "slow" updates (or any updates for that matter),
keep track of the current (aka "expected") _version_ field of the solr
document when your updater starts processing -- and pass that in along       
with the new update -- solr will reject an update if the specified
_version_ doesn't match what's in the index.

https://lucene.apache.org/solr/guide/updating-parts-of-documents.html#optimistic-concurrency

So imagine the current instock=1 version of your product is 42, and you
start a "slow" update to change the "name" field ... while that's in 
progress a "fast" update sets instock=0 and now you have a new 
_version_=666.  When the "slow" updater is done building up the entire 
document, and sends it to solr along with the _version_=42 assumption, 
solr will reject the update with a "Conflict (409)" HTTP Status, and your 
slow update code can say "ok ... i must have stale data, let's try again"



: 
: -----Original Message-----
: From: Erick Erickson [mailto:erickerickson@gmail.com] 
: Sent: Monday, February 12, 2018 11:17 AM
: To: solr-user <solr-user@lucene.apache.org>
: Subject: Re: DovValues and in-place udpates
: 
: "But it also triggers a slow update that will rebuild the entire document..."
: 
: Why do you think this? The whole _point_ of in-place updates is that they don't have to
re-index the whole document.... And the only way to do that effectively would be if all the
fields are stored, which is not a requirement for in-place updates.
: 
: Best,
: Erick
: 
: On Mon, Feb 12, 2018 at 8:02 AM, Brian Yee <byee@wayfair.com> wrote:
: > I asked a question here about fast inventory updates last week and I was recommended
to use docValues with partial in-place updates. I think this will work well, but there is
a problem I can't think of a good solution for.
: >
: > Consider this scenario:
: > InStock = 1 for a product.
: > InStock changes to 0 which triggers a fast in-place update with docValues.
: > But it also triggers a slow update that will rebuild the entire document. Let's say
that takes 10 minutes because we do updates in batches.
: > During that 5 minutes, InStock changes again to 1 which triggers a fast update to solr.
So in Solr InStock=1 which is correct.
: > The slow update finishes and overwrites InStock=0 which is incorrect.
: >
: > How can we deal with this situation?
: 

-Hoss
http://www.lucidworks.com/

Mime
View raw message