lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (SOLR-11882) SolrMetric registries retain references to SolrCores when closed
Date Thu, 01 Feb 2018 17:15:00 GMT


ASF subversion and git services commented on SOLR-11882:

Commit 8418081c4ae5bfe752938c1ae6db9cf5063c8e7f in lucene-solr's branch refs/heads/master
from [~ab]
[;h=8418081 ]

Revert "SOLR-11882: SolrMetric registries retain references to SolrCores when closed"

This reverts commit f0509c19c16ded1557f8d7168acb0b7faf926ab7.

> SolrMetric registries retain references to SolrCores when closed
> ----------------------------------------------------------------
>                 Key: SOLR-11882
>                 URL:
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: metrics, Server
>    Affects Versions: 7.1
>            Reporter: Eros Taborelli
>            Assignee: Erick Erickson
>            Priority: Major
>             Fix For: 7.3
>         Attachments: SOLR-11882.patch, SOLR-11882.patch, SOLR-11882.patch, SOLR-11882.patch,,,
> *Description:*
> Our setup involves using a lot of small cores (possibly hundred thousand), but working
only on a few of them at any given time.
> We already followed all recommendations in this guide: []
> We noticed that after creating/loading around 1000-2000 empty cores, with no documents
inside, the heap consumption went through the roof despite having set transientCacheSize to
only 64 (heap size set to 12G).
> All cores are correctly set to loadOnStartup=false and transient=true, and we have verified
via logs that the cores in excess are actually being closed.
> However, a reference remains in the org.apache.solr.metrics.SolrMetricManager#registries
that is never removed until a core if fully unloaded.
> Restarting the JVM loads all cores in the admin UI, but doesn't populate the ConcurrentHashMap
until a core is actually fully loaded.
> I reproduced the issue on a smaller scale (transientCacheSize = 5, heap size = 512m)
and made a report (attached) using eclipse MAT.
> *Desired outcome:*
> When a transient core is closed, the references in the SolrMetricManager should be removed,
in the same fashion the reporters for the core are also closed and removed.
> In alternative, a unloadOnClose=true|false flag could be implemented to fully unload
a transient core when closed due to the cache size.
> *Note:*
> The documentation mentions everywhere that the unused cores will be unloaded, but it's
misleading as the cores are never fully unloaded.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message