lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gus Heck (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SOLR-8349) Allow sharing of large in memory data structures across cores
Date Wed, 17 Feb 2016 21:21:18 GMT

     [ https://issues.apache.org/jira/browse/SOLR-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gus Heck updated SOLR-8349:
---------------------------
    Attachment: SOLR-8349.patch

Changes since original patch:
# No interface added to lucene, and no support for lucene analyzers to use this feature at
this time. (defer to subsequent enhancement)
# Caching now done using a Guava cache. This simplifies code, but means that a core loading
a given resource will now block the loading of other cores that also require that resource.
# As suggested by Dave, the cache now uses weak references to the values avoiding the possibility
of a ClassLoader memory leak. The SolrCore object now gets a hard reference to the object.
This means that there is now some chance that a resource will be unloaded if the last core
using it is unloaded and then loaded again. This differs from the previous code where it was
guaranteed to not be unloaded. To guarantee an deterministic behavior we likely would need
to use hard references in the cache as before, or add something along the lines of a reference
counting scheme to know when the last core stopped using it.
# To ensure that the SolrCore is given a reference to the loaded object, a ContainerResourceAware
interface was added and is treated similarly to SolrCoreAware and ResourceLoaderAware. This
allows components to give us the loading code before we give them the core, and thus we can
coordinate the loading such that no there is always a hard reference to the resource (until
the last core using it is unloaded). 

> Allow sharing of large in memory data structures across cores
> -------------------------------------------------------------
>
>                 Key: SOLR-8349
>                 URL: https://issues.apache.org/jira/browse/SOLR-8349
>             Project: Solr
>          Issue Type: Improvement
>          Components: Server
>    Affects Versions: 5.3
>            Reporter: Gus Heck
>         Attachments: SOLR-8349.patch, SOLR-8349.patch
>
>
> In some cases search components or analysis classes may utilize a large dictionary or
other in-memory structure. When multiple cores are loaded with identical configurations utilizing
this large in memory structure, each core holds it's own copy in memory. This has been noted
in the past and a specific case reported in SOLR-3443. This patch provides a generalized capability,
and if accepted, this capability will then be used to fix SOLR-3443.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message