lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Atomic Updates
Date Thu, 27 Apr 2017 14:53:48 GMT
bq: but is there any possibility that the values stick around until
there is a segment merge for some strange reason

There better not be or it's a bug. Things will stick around until
you issue a commit, is there any chance that's the problem?

If you can document the exact steps, maybe we can reproduce
the issue and raise a JIRA.

Best,
Erick

On Thu, Apr 27, 2017 at 6:03 AM, Chris Ulicny <culicny@iq.media> wrote:
> Yeah, something's not quite right somewhere. We never even considered
> in-place updates an option since it requires the fields to be non-indexed
> and non-stored. Our schemas never have any field that satisfies those two
> conditions let alone the other necessary ones.
>
> I went ahead and tested the atomic updates on different textfields, and I
> still can't get the indexed but not-stored othertext_field to disappear. So
> far set, add, and remove updates do not change it regardless of what the
> fields are in the atomic update.
>
> It would be extraordinarily useful if this update behavior is now expected
> (but not currently documented) functionality.
>
> I'm not too familiar with the nitty-gritty details of merging segment
> files, but is there any possibility that the values stick around until
> there is a segment merge for some strange reason?
>
> On Thu, Apr 27, 2017 at 12:59 AM Dorian Hoxha <dorian.hoxha@gmail.com>
> wrote:
>
>> @Chris,
>> According to doc-link-above, only INC,SET are in-place-updates. And only
>> when they're not indexed/stored, while your 'integer-field' is. So still
>> shenanigans in there somewhere (docs,your-code,your-test,solr-code).
>>
>> On Thu, Apr 27, 2017 at 2:04 AM, Chris Ulicny <culicny@iq.media> wrote:
>>
>> > That's probably it then. None of the atomic updates that I've tried have
>> > been on TextFields. I'll give the TextField atomic update to verify that
>> it
>> > will clear the other field.
>> >
>> > Has this functionality been consistent since atomic updates were
>> > introduced, or is this a side effect of some other change? It'd be very
>> > convenient for us to use this functionality as it currently works, but if
>> > it's something that prevents us from upgrading versions in the future, we
>> > should probably avoid expecting it to work.
>> >
>> > On Wed, Apr 26, 2017 at 7:36 PM Ishan Chattopadhyaya <
>> > ichattopadhyaya@gmail.com> wrote:
>> >
>> > > > Hmm, interesting. I can imagine that as long as you're updating
>> > > > docValues fields, the other_text field would be there. But the
>> instant
>> > > > you updated a non-docValues field (text_field in your example) the
>> > > > other_text field would disappear
>> > >
>> > > I can confirm this. When in-place updates to DV fields are done, the
>> rest
>> > > of the fields remain as they were.
>> > >
>> > > On Thu, Apr 27, 2017 at 4:33 AM, Erick Erickson <
>> erickerickson@gmail.com
>> > >
>> > > wrote:
>> > >
>> > > > Hmm, interesting. I can imagine that as long as you're updating
>> > > > docValues fields, the other_text field would be there. But the
>> instant
>> > > > you updated a non-docValues field (text_field in your example) the
>> > > > other_text field would disappear.
>> > > >
>> > > > I DO NOT KNOW this for a fact, but I'm asking people who do.
>> > > >
>> > > > On Wed, Apr 26, 2017 at 2:13 PM, Dorian Hoxha <
>> dorian.hoxha@gmail.com>
>> > > > wrote:
>> > > > > There are In Place Updates, but according to docs they stll
>> shouldn't
>> > > > work
>> > > > > in your case:
>> > > > > https://cwiki.apache.org/confluence/display/solr/
>> > > > Updating+Parts+of+Documents
>> > > > >
>> > > > > On Wed, Apr 26, 2017 at 10:36 PM, Chris Ulicny <culicny@iq.media>
>> > > wrote:
>> > > > >
>> > > > >> That's the thing I'm curious about though. As I mentioned
in the
>> > first
>> > > > >> post, I've already tried a few tests, and the value seems
to still
>> > be
>> > > > >> present after an atomic update.
>> > > > >>
>> > > > >> I haven't exhausted all possible atomic updates, but 'set'
and
>> 'add'
>> > > > seem
>> > > > >> to preserve the non-stored text field.
>> > > > >>
>> > > > >> Thanks,
>> > > > >> Chris
>> > > > >>
>> > > > >> On Wed, Apr 26, 2017 at 4:07 PM Dorian Hoxha <
>> > dorian.hoxha@gmail.com>
>> > > > >> wrote:
>> > > > >>
>> > > > >> > You'll lose the data in that field. Try doing a commit
and it
>> > should
>> > > > >> > happen.
>> > > > >> >
>> > > > >> > On Wed, Apr 26, 2017 at 9:50 PM, Chris Ulicny <culicny@iq.media
>> >
>> > > > wrote:
>> > > > >> >
>> > > > >> > > Thanks Shawn, I didn't realize docValues were enabled
by
>> default
>> > > > now.
>> > > > >> > > That's very convenient and probably makes a lot
of the schemas
>> > > we've
>> > > > >> been
>> > > > >> > > making excessively verbose.
>> > > > >> > >
>> > > > >> > > This is on 6.3.0. Do you know what the first version
was that
>> > they
>> > > > >> added
>> > > > >> > > the docValues by default for non-Text field?
>> > > > >> > >
>> > > > >> > > However, that shouldn't apply to this since I'm
concerned
>> with a
>> > > > >> > non-stored
>> > > > >> > > TextField without docValues enabled.
>> > > > >> > >
>> > > > >> > > Best,
>> > > > >> > > Chris
>> > > > >> > >
>> > > > >> > > On Wed, Apr 26, 2017 at 3:36 PM Shawn Heisey <
>> > apache@elyograg.org
>> > > >
>> > > > >> > wrote:
>> > > > >> > >
>> > > > >> > > > On 4/25/2017 1:40 PM, Chris Ulicny wrote:
>> > > > >> > > > > Hello all,
>> > > > >> > > > >
>> > > > >> > > > > Suppose I have the following fields in
a document and
>> > populate
>> > > > all
>> > > > >> 4
>> > > > >> > > > fields
>> > > > >> > > > > for every document.
>> > > > >> > > > >
>> > > > >> > > > > id: uniqueKey, indexed and stored
>> > > > >> > > > > integer_field: indexed and stored
>> > > > >> > > > > text_field: indexed and stored
>> > > > >> > > > > othertext_field: indexed but not stored
>> > > > >> > > > >
>> > > > >> > > > > No default values, multivalues, docvalues,
copyfields, or
>> > any
>> > > > other
>> > > > >> > > > > properties set.
>> > > > >> > > >
>> > > > >> > > > You didn't indicate the Solr version.  In
recent Solr
>> > versions,
>> > > > most
>> > > > >> > > > field classes other than TextField have docValues
enabled by
>> > > > default,
>> > > > >> > > > even if the config is not mentioned on the
field, and in
>> those
>> > > > >> > versions,
>> > > > >> > > > docValues will take the place of stored if
stored is false.
>> > > > >> > > >
>> > > > >> > > > Thanks,
>> > > > >> > > > Shawn
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>>

Mime
View raw message