lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Ulicny <culi...@iq.media>
Subject Re: Atomic Updates
Date Thu, 27 Apr 2017 15:23:13 GMT
I'm sending commit=true with every update while testing. I'll write up the
tests and see if someone else can reproduce it.

On Thu, Apr 27, 2017 at 10:54 AM Erick Erickson <erickerickson@gmail.com>
wrote:

> bq: but is there any possibility that the values stick around until
> there is a segment merge for some strange reason
>
> There better not be or it's a bug. Things will stick around until
> you issue a commit, is there any chance that's the problem?
>
> If you can document the exact steps, maybe we can reproduce
> the issue and raise a JIRA.
>
> Best,
> Erick
>
> On Thu, Apr 27, 2017 at 6:03 AM, Chris Ulicny <culicny@iq.media> wrote:
> > Yeah, something's not quite right somewhere. We never even considered
> > in-place updates an option since it requires the fields to be non-indexed
> > and non-stored. Our schemas never have any field that satisfies those two
> > conditions let alone the other necessary ones.
> >
> > I went ahead and tested the atomic updates on different textfields, and I
> > still can't get the indexed but not-stored othertext_field to disappear.
> So
> > far set, add, and remove updates do not change it regardless of what the
> > fields are in the atomic update.
> >
> > It would be extraordinarily useful if this update behavior is now
> expected
> > (but not currently documented) functionality.
> >
> > I'm not too familiar with the nitty-gritty details of merging segment
> > files, but is there any possibility that the values stick around until
> > there is a segment merge for some strange reason?
> >
> > On Thu, Apr 27, 2017 at 12:59 AM Dorian Hoxha <dorian.hoxha@gmail.com>
> > wrote:
> >
> >> @Chris,
> >> According to doc-link-above, only INC,SET are in-place-updates. And only
> >> when they're not indexed/stored, while your 'integer-field' is. So still
> >> shenanigans in there somewhere (docs,your-code,your-test,solr-code).
> >>
> >> On Thu, Apr 27, 2017 at 2:04 AM, Chris Ulicny <culicny@iq.media> wrote:
> >>
> >> > That's probably it then. None of the atomic updates that I've tried
> have
> >> > been on TextFields. I'll give the TextField atomic update to verify
> that
> >> it
> >> > will clear the other field.
> >> >
> >> > Has this functionality been consistent since atomic updates were
> >> > introduced, or is this a side effect of some other change? It'd be
> very
> >> > convenient for us to use this functionality as it currently works,
> but if
> >> > it's something that prevents us from upgrading versions in the
> future, we
> >> > should probably avoid expecting it to work.
> >> >
> >> > On Wed, Apr 26, 2017 at 7:36 PM Ishan Chattopadhyaya <
> >> > ichattopadhyaya@gmail.com> wrote:
> >> >
> >> > > > Hmm, interesting. I can imagine that as long as you're updating
> >> > > > docValues fields, the other_text field would be there. But the
> >> instant
> >> > > > you updated a non-docValues field (text_field in your example)
the
> >> > > > other_text field would disappear
> >> > >
> >> > > I can confirm this. When in-place updates to DV fields are done, the
> >> rest
> >> > > of the fields remain as they were.
> >> > >
> >> > > On Thu, Apr 27, 2017 at 4:33 AM, Erick Erickson <
> >> erickerickson@gmail.com
> >> > >
> >> > > wrote:
> >> > >
> >> > > > Hmm, interesting. I can imagine that as long as you're updating
> >> > > > docValues fields, the other_text field would be there. But the
> >> instant
> >> > > > you updated a non-docValues field (text_field in your example)
the
> >> > > > other_text field would disappear.
> >> > > >
> >> > > > I DO NOT KNOW this for a fact, but I'm asking people who do.
> >> > > >
> >> > > > On Wed, Apr 26, 2017 at 2:13 PM, Dorian Hoxha <
> >> dorian.hoxha@gmail.com>
> >> > > > wrote:
> >> > > > > There are In Place Updates, but according to docs they stll
> >> shouldn't
> >> > > > work
> >> > > > > in your case:
> >> > > > > https://cwiki.apache.org/confluence/display/solr/
> >> > > > Updating+Parts+of+Documents
> >> > > > >
> >> > > > > On Wed, Apr 26, 2017 at 10:36 PM, Chris Ulicny <culicny@iq.media
> >
> >> > > wrote:
> >> > > > >
> >> > > > >> That's the thing I'm curious about though. As I mentioned
in
> the
> >> > first
> >> > > > >> post, I've already tried a few tests, and the value
seems to
> still
> >> > be
> >> > > > >> present after an atomic update.
> >> > > > >>
> >> > > > >> I haven't exhausted all possible atomic updates, but
'set' and
> >> 'add'
> >> > > > seem
> >> > > > >> to preserve the non-stored text field.
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >> Chris
> >> > > > >>
> >> > > > >> On Wed, Apr 26, 2017 at 4:07 PM Dorian Hoxha <
> >> > dorian.hoxha@gmail.com>
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >> > You'll lose the data in that field. Try doing a
commit and it
> >> > should
> >> > > > >> > happen.
> >> > > > >> >
> >> > > > >> > On Wed, Apr 26, 2017 at 9:50 PM, Chris Ulicny
> <culicny@iq.media
> >> >
> >> > > > wrote:
> >> > > > >> >
> >> > > > >> > > Thanks Shawn, I didn't realize docValues were
enabled by
> >> default
> >> > > > now.
> >> > > > >> > > That's very convenient and probably makes
a lot of the
> schemas
> >> > > we've
> >> > > > >> been
> >> > > > >> > > making excessively verbose.
> >> > > > >> > >
> >> > > > >> > > This is on 6.3.0. Do you know what the first
version was
> that
> >> > they
> >> > > > >> added
> >> > > > >> > > the docValues by default for non-Text field?
> >> > > > >> > >
> >> > > > >> > > However, that shouldn't apply to this since
I'm concerned
> >> with a
> >> > > > >> > non-stored
> >> > > > >> > > TextField without docValues enabled.
> >> > > > >> > >
> >> > > > >> > > Best,
> >> > > > >> > > Chris
> >> > > > >> > >
> >> > > > >> > > On Wed, Apr 26, 2017 at 3:36 PM Shawn Heisey
<
> >> > apache@elyograg.org
> >> > > >
> >> > > > >> > wrote:
> >> > > > >> > >
> >> > > > >> > > > On 4/25/2017 1:40 PM, Chris Ulicny wrote:
> >> > > > >> > > > > Hello all,
> >> > > > >> > > > >
> >> > > > >> > > > > Suppose I have the following fields
in a document and
> >> > populate
> >> > > > all
> >> > > > >> 4
> >> > > > >> > > > fields
> >> > > > >> > > > > for every document.
> >> > > > >> > > > >
> >> > > > >> > > > > id: uniqueKey, indexed and stored
> >> > > > >> > > > > integer_field: indexed and stored
> >> > > > >> > > > > text_field: indexed and stored
> >> > > > >> > > > > othertext_field: indexed but not
stored
> >> > > > >> > > > >
> >> > > > >> > > > > No default values, multivalues,
docvalues, copyfields,
> or
> >> > any
> >> > > > other
> >> > > > >> > > > > properties set.
> >> > > > >> > > >
> >> > > > >> > > > You didn't indicate the Solr version.
 In recent Solr
> >> > versions,
> >> > > > most
> >> > > > >> > > > field classes other than TextField have
docValues
> enabled by
> >> > > > default,
> >> > > > >> > > > even if the config is not mentioned on
the field, and in
> >> those
> >> > > > >> > versions,
> >> > > > >> > > > docValues will take the place of stored
if stored is
> false.
> >> > > > >> > > >
> >> > > > >> > > > Thanks,
> >> > > > >> > > > Shawn
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message