lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emir Arnautovic <emir.arnauto...@sematext.com>
Subject Re: in-place atomic updates for numeric docValue field
Date Fri, 05 May 2017 08:34:51 GMT
Hi Dan,

In-place updates are working because index size does not change. Atomic 
(or any other updates) are flagging existing doc as deleted and writing 
it again, so even if it removes some fields, such updates are making 
index larger until segment with deleted doc is merged.

In-place updates are making changes of existing doc values file - think 
of it as 4B that are updated - some value has to be set. Removing it 
would require all bytes after to be shifted = creating new file. That's 
why in-place updates work only for numeric fields (fixed width) and 
supports only set and inc.

Emir


On 04.05.2017 16:57, Dan . wrote:
> Hi Emir,
>
> Yes I though of representing -1 as null, but  this makes the index
> unnecessarily larger, particularly if we have to default all docs to this
> value.
>
> Cheers,
> Dan
>
> On 4 May 2017 at 15:16, Emir Arnautovic <emir.arnautovic@sematext.com>
> wrote:
>
>> Hi Dan,
>>
>> Remove does not make sense when it comes to in-place updates of docValues
>> - it has to have some value, so only thing that you can do is introduce
>> some int value as null.
>>
>> HTH,
>> Emir
>>
>>
>>
>> On 04.05.2017 15:40, Dan . wrote:
>>
>>> Hi,
>>>
>>> I have a field like this:
>>>
>>> <fieldType name="integer" class="solr.TrieIntField" omitNorms="true"/>
>>> <field name="popularity" type="integer" indexed="false" stored="false"
>>> docValues="true" multiValued="false"/>
>>>
>>> so I can do a fast in-place atomic updates
>>>
>>> However if I do e.g.
>>>
>>> curl -H 'Content-Type: application/json'
>>> 'http://localhost:8983/solr/collection/update?commit=true'
>>> --data-binary '
>>> [{
>>>    "id":"my_id",
>>>    "popularity":{"set":null}
>>> }]'
>>>
>>> then I'd expect the popularity field to be removed, however it's not.
>>>
>>> I this a bug? or is there a know workaround for this for in-place atomic
>>> updates?
>>>
>>> Cheers,
>>> Dan
>>>
>>>
>> --
>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
>> Solr & Elasticsearch Support * http://sematext.com/
>>
>>

-- 
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/


Mime
View raw message