lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ishan Chattopadhyaya (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-5944) Support updates of numeric DocValues
Date Sun, 22 Jan 2017 16:25:27 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833574#comment-15833574
] 

Ishan Chattopadhyaya edited comment on SOLR-5944 at 1/22/17 4:24 PM:
---------------------------------------------------------------------

Hoss did some initial single threaded benchmarks on the performance impact of doing in-place
updates as compared to regular atomic updates.
{code}
The script [0] adds 20K docs containing a stored+indexed text field, a DVO long field, and
a stored+indexed long field. It then does 50 iterations of 5K updates to each of the 2 long
fields (500K updates total, 250K on the DVO field, 250K on the stored+indexed field), recording
the cumulative total times for the udpates to each type of field.
{code}

Here are some results (times are in seconds):
Hoss' run, as of c21d8a005387eae4d0f0adde4de7e4e465fb73c8, on his laptop:
||Branch||Adds||DVO Updates||stored+indexed Updates||
|master|52|531|543|
|5944|51|352|503|

My run, as of right now:
||Branch||Adds||DVO Updates||stored+indexed Updates||
|master|40|682|663|
|5944|36|295|694|

Seems like Hoss observed a 30% speedup on 5944 branch (in-place update vs. regular update),
and I observed a 57% speedup on the same usecase. More benchmarks need to done to analyse
the performance in multi-threaded indexing, and also to ensure there's no performance regression
for non-dv updates.

 [0] - https://gist.github.com/anonymous/ae8bf7db3c713bf9d45937ec0aa1cfae


was (Author: ichattopadhyaya):
Hoss did some initial single threaded benchmarks on the performance impact of doing in-place
updates as compared to regular atomic updates.
{code}
The script [0] adds 20K docs containing a stored+indexed text field, a DVO long field, and
a stored+indexed text field. It then does 50 iterations of 5K updates to each of the 2 long
fields (500K updates total, 250K on the DVO field, 250K on the stored+indexed field), recording
the cumulative total times for the udpates to each type of field.
{code}

Here are some results (times are in seconds):
Hoss' run, as of c21d8a005387eae4d0f0adde4de7e4e465fb73c8, on his laptop:
||Branch||Adds||DVO Updates||stored+indexed Updates||
|master|52|531|543|
|5944|51|352|503|

My run, as of right now:
||Branch||Adds||DVO Updates||stored+indexed Updates||
|master|40|682|663|
|5944|36|295|694|

Seems like Hoss observed a 30% speedup on 5944 branch (in-place update vs. regular update),
and I observed a 57% speedup on the same usecase. More benchmarks need to done to analyse
the performance in multi-threaded indexing, and also to ensure there's no performance regression
for non-dv updates.

 [0] - https://gist.github.com/anonymous/ae8bf7db3c713bf9d45937ec0aa1cfae

> Support updates of numeric DocValues
> ------------------------------------
>
>                 Key: SOLR-5944
>                 URL: https://issues.apache.org/jira/browse/SOLR-5944
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Ishan Chattopadhyaya
>            Assignee: Shalin Shekhar Mangar
>         Attachments: defensive-checks.log.gz, demo-why-dynamic-fields-cannot-be-inplace-updated-first-time.patch,
DUP.patch, hoss.62D328FA1DEA57FD.fail2.txt, hoss.62D328FA1DEA57FD.fail3.txt, hoss.62D328FA1DEA57FD.fail.txt,
hoss.D768DD9443A98DC.fail.txt, hoss.D768DD9443A98DC.pass.txt, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt,
TestStressInPlaceUpdates.eb044ac71.failures.tar.gz
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be really nice
to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message