lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (SOLR-9579) Reuse lucene FieldType in createField flow during ingestion
Date Tue, 25 Oct 2016 21:30:58 GMT


ASF subversion and git services commented on SOLR-9579:

Commit 941c5e92ba6ff76e913746caf68e05b563983f17 in lucene-solr's branch refs/heads/master
from [~steve_rowe]
[;h=941c5e9 ]

SOLR-9579: fix intellij compilation: add lucene core dependency to the langid contrib

> Reuse lucene FieldType in createField flow during ingestion
> -----------------------------------------------------------
>                 Key: SOLR-9579
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>          Components: Schema and Analysis
>    Affects Versions: 6.x, master (7.0)
>         Environment: This has been primarily tested on Windows 8 and Windows Server 2012
>            Reporter: John Call
>            Priority: Minor
>              Labels: gc, memory, reuse
>             Fix For: master (7.0)
>         Attachments: SOLR-9579.patch, SOLR-9579.patch
>   Original Estimate: 2h
>  Remaining Estimate: 2h
> During ingestion createField in FieldType is being called for each field on each document.
For the subclasses of FieldType without their own implementation of createField the lucene
version of FieldType is created to be stored along with the value. However the lucene FieldType
object is identical when created from the same SchemaField. In testing ingestion of one million
rows with 22 field each we were creating 22 million lucene FieldType objects when only 22
are needed. Solr should lazily initialize a lucene FieldType for each SchemaField and reuse
them for future ingestion. Not only does this relieve memory usage but also relieves significant
pressure on the gc.
> There are also subclasses of Solr FieldType which create separate Lucene FieldType for
stored fields instead of reusing the static in StoredField.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message