lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5714) You should be able to use one pool of memory for multiple collection's HDFS block caches.
Date Tue, 04 Mar 2014 01:26:23 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918888#comment-13918888
] 

Mark Miller commented on SOLR-5714:
-----------------------------------

To add a bit: I wanted to do this even before realizing the same index issue - because these
caches allocate the memory up front, users have already been running into issues when trying
to use multiple collections. The same index issue just made matters even worse because it
wouldn't like be an issue until the worse time - during recovery.

By and large though, I think we will have fewer user issues if they can simply decide up front
how much RAM they can dedicate to HDFS caching and let Solr work out the rest across their
collections. It's more dynamic and takes a large burden off the user.

> You should be able to use one pool of memory for multiple collection's HDFS block caches.
> -----------------------------------------------------------------------------------------
>
>                 Key: SOLR-5714
>                 URL: https://issues.apache.org/jira/browse/SOLR-5714
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Mark Miller
>            Assignee: Mark Miller
>             Fix For: 4.8, 5.0
>
>         Attachments: SOLR-5714.patch, SOLR-5714.patch, SOLR-5714.patch, SOLR-5714.patch
>
>
> Currently, you have to specify how much direct memory to allocate per SolrCore. This
can be inefficient, and has some negative consequences - for instance, when replicating, many
times two HDFS directories will exist for the same index briefly, which will double the RAM
used for that SolrCore.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message