lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Ulicny <culi...@iq.media>
Subject Re: Atomic Updates
Date Thu, 27 Apr 2017 17:29:54 GMT
While recreating it with a fresh schema, I realized that this was a case of
a very, very stupid user error during configuring the cores.

I setup the testing cores with the wrong configset, and then proceeded to
edit the schema in the right configset. So, the field was actually stored
by default, but I wasn't attempting to retrieve it so I never realized it
was being stored since I ended up looking at the wrong schema.

I switched the config sets and everything works as expected. Any atomic
updates clear out the indexed values for the non-stored field.

Thanks for bearing with me.
Chris


On Thu, Apr 27, 2017 at 11:23 AM Chris Ulicny <culicny@iq.media> wrote:

> I'm sending commit=true with every update while testing. I'll write up the
> tests and see if someone else can reproduce it.
>
> On Thu, Apr 27, 2017 at 10:54 AM Erick Erickson <erickerickson@gmail.com>
> wrote:
>
>> bq: but is there any possibility that the values stick around until
>> there is a segment merge for some strange reason
>>
>> There better not be or it's a bug. Things will stick around until
>> you issue a commit, is there any chance that's the problem?
>>
>> If you can document the exact steps, maybe we can reproduce
>> the issue and raise a JIRA.
>>
>> Best,
>> Erick
>>
>> On Thu, Apr 27, 2017 at 6:03 AM, Chris Ulicny <culicny@iq.media> wrote:
>> > Yeah, something's not quite right somewhere. We never even considered
>> > in-place updates an option since it requires the fields to be
>> non-indexed
>> > and non-stored. Our schemas never have any field that satisfies those
>> two
>> > conditions let alone the other necessary ones.
>> >
>> > I went ahead and tested the atomic updates on different textfields, and
>> I
>> > still can't get the indexed but not-stored othertext_field to
>> disappear. So
>> > far set, add, and remove updates do not change it regardless of what the
>> > fields are in the atomic update.
>> >
>> > It would be extraordinarily useful if this update behavior is now
>> expected
>> > (but not currently documented) functionality.
>> >
>> > I'm not too familiar with the nitty-gritty details of merging segment
>> > files, but is there any possibility that the values stick around until
>> > there is a segment merge for some strange reason?
>> >
>> > On Thu, Apr 27, 2017 at 12:59 AM Dorian Hoxha <dorian.hoxha@gmail.com>
>> > wrote:
>> >
>> >> @Chris,
>> >> According to doc-link-above, only INC,SET are in-place-updates. And
>> only
>> >> when they're not indexed/stored, while your 'integer-field' is. So
>> still
>> >> shenanigans in there somewhere (docs,your-code,your-test,solr-code).
>> >>
>> >> On Thu, Apr 27, 2017 at 2:04 AM, Chris Ulicny <culicny@iq.media>
>> wrote:
>> >>
>> >> > That's probably it then. None of the atomic updates that I've tried
>> have
>> >> > been on TextFields. I'll give the TextField atomic update to verify
>> that
>> >> it
>> >> > will clear the other field.
>> >> >
>> >> > Has this functionality been consistent since atomic updates were
>> >> > introduced, or is this a side effect of some other change? It'd be
>> very
>> >> > convenient for us to use this functionality as it currently works,
>> but if
>> >> > it's something that prevents us from upgrading versions in the
>> future, we
>> >> > should probably avoid expecting it to work.
>> >> >
>> >> > On Wed, Apr 26, 2017 at 7:36 PM Ishan Chattopadhyaya <
>> >> > ichattopadhyaya@gmail.com> wrote:
>> >> >
>> >> > > > Hmm, interesting. I can imagine that as long as you're updating
>> >> > > > docValues fields, the other_text field would be there. But
the
>> >> instant
>> >> > > > you updated a non-docValues field (text_field in your example)
>> the
>> >> > > > other_text field would disappear
>> >> > >
>> >> > > I can confirm this. When in-place updates to DV fields are done,
>> the
>> >> rest
>> >> > > of the fields remain as they were.
>> >> > >
>> >> > > On Thu, Apr 27, 2017 at 4:33 AM, Erick Erickson <
>> >> erickerickson@gmail.com
>> >> > >
>> >> > > wrote:
>> >> > >
>> >> > > > Hmm, interesting. I can imagine that as long as you're updating
>> >> > > > docValues fields, the other_text field would be there. But
the
>> >> instant
>> >> > > > you updated a non-docValues field (text_field in your example)
>> the
>> >> > > > other_text field would disappear.
>> >> > > >
>> >> > > > I DO NOT KNOW this for a fact, but I'm asking people who
do.
>> >> > > >
>> >> > > > On Wed, Apr 26, 2017 at 2:13 PM, Dorian Hoxha <
>> >> dorian.hoxha@gmail.com>
>> >> > > > wrote:
>> >> > > > > There are In Place Updates, but according to docs they
stll
>> >> shouldn't
>> >> > > > work
>> >> > > > > in your case:
>> >> > > > > https://cwiki.apache.org/confluence/display/solr/
>> >> > > > Updating+Parts+of+Documents
>> >> > > > >
>> >> > > > > On Wed, Apr 26, 2017 at 10:36 PM, Chris Ulicny
>> <culicny@iq.media>
>> >> > > wrote:
>> >> > > > >
>> >> > > > >> That's the thing I'm curious about though. As I
mentioned in
>> the
>> >> > first
>> >> > > > >> post, I've already tried a few tests, and the value
seems to
>> still
>> >> > be
>> >> > > > >> present after an atomic update.
>> >> > > > >>
>> >> > > > >> I haven't exhausted all possible atomic updates,
but 'set' and
>> >> 'add'
>> >> > > > seem
>> >> > > > >> to preserve the non-stored text field.
>> >> > > > >>
>> >> > > > >> Thanks,
>> >> > > > >> Chris
>> >> > > > >>
>> >> > > > >> On Wed, Apr 26, 2017 at 4:07 PM Dorian Hoxha <
>> >> > dorian.hoxha@gmail.com>
>> >> > > > >> wrote:
>> >> > > > >>
>> >> > > > >> > You'll lose the data in that field. Try doing
a commit and
>> it
>> >> > should
>> >> > > > >> > happen.
>> >> > > > >> >
>> >> > > > >> > On Wed, Apr 26, 2017 at 9:50 PM, Chris Ulicny
>> <culicny@iq.media
>> >> >
>> >> > > > wrote:
>> >> > > > >> >
>> >> > > > >> > > Thanks Shawn, I didn't realize docValues
were enabled by
>> >> default
>> >> > > > now.
>> >> > > > >> > > That's very convenient and probably makes
a lot of the
>> schemas
>> >> > > we've
>> >> > > > >> been
>> >> > > > >> > > making excessively verbose.
>> >> > > > >> > >
>> >> > > > >> > > This is on 6.3.0. Do you know what the
first version was
>> that
>> >> > they
>> >> > > > >> added
>> >> > > > >> > > the docValues by default for non-Text
field?
>> >> > > > >> > >
>> >> > > > >> > > However, that shouldn't apply to this
since I'm concerned
>> >> with a
>> >> > > > >> > non-stored
>> >> > > > >> > > TextField without docValues enabled.
>> >> > > > >> > >
>> >> > > > >> > > Best,
>> >> > > > >> > > Chris
>> >> > > > >> > >
>> >> > > > >> > > On Wed, Apr 26, 2017 at 3:36 PM Shawn
Heisey <
>> >> > apache@elyograg.org
>> >> > > >
>> >> > > > >> > wrote:
>> >> > > > >> > >
>> >> > > > >> > > > On 4/25/2017 1:40 PM, Chris Ulicny
wrote:
>> >> > > > >> > > > > Hello all,
>> >> > > > >> > > > >
>> >> > > > >> > > > > Suppose I have the following
fields in a document and
>> >> > populate
>> >> > > > all
>> >> > > > >> 4
>> >> > > > >> > > > fields
>> >> > > > >> > > > > for every document.
>> >> > > > >> > > > >
>> >> > > > >> > > > > id: uniqueKey, indexed and stored
>> >> > > > >> > > > > integer_field: indexed and stored
>> >> > > > >> > > > > text_field: indexed and stored
>> >> > > > >> > > > > othertext_field: indexed but
not stored
>> >> > > > >> > > > >
>> >> > > > >> > > > > No default values, multivalues,
docvalues,
>> copyfields, or
>> >> > any
>> >> > > > other
>> >> > > > >> > > > > properties set.
>> >> > > > >> > > >
>> >> > > > >> > > > You didn't indicate the Solr version.
 In recent Solr
>> >> > versions,
>> >> > > > most
>> >> > > > >> > > > field classes other than TextField
have docValues
>> enabled by
>> >> > > > default,
>> >> > > > >> > > > even if the config is not mentioned
on the field, and in
>> >> those
>> >> > > > >> > versions,
>> >> > > > >> > > > docValues will take the place of
stored if stored is
>> false.
>> >> > > > >> > > >
>> >> > > > >> > > > Thanks,
>> >> > > > >> > > > Shawn
>> >> > > > >> > > >
>> >> > > > >> > > >
>> >> > > > >> > >
>> >> > > > >> >
>> >> > > > >>
>> >> > > >
>> >> > >
>> >> >
>> >>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message