lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gus Heck (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-8349) Allow sharing of large in memory data structures across cores
Date Mon, 22 Feb 2016 00:24:18 GMT

    [ https://issues.apache.org/jira/browse/SOLR-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15156315#comment-15156315
] 

Gus Heck commented on SOLR-8349:
--------------------------------

I do see how what you suggest could be made to work if a decoder was supplied and the decoded
version was what was cached. 

So are you proposing changing JarRepository, or adding a similar class called BlobRepository?
JarRepository does seem very Jar focused. (returning JarContent, JarContentRef, etc..) It
would for example no longer be possible to return a ByteBuffer type from getByteBuffer, since
returning the bytes from which to decode a HashMap won't help. Seems like fairly major surgery
is required to make the JarRepository class fully generic. 

I do like the versioned resource and config API bit though, that's nice.

I need to better understand the lazy="true" bit you mentioned, because I very much don't ever
want a user query to be the thing that causes the resource to load (a process that could take
minutes). The resource should be available and ready to go when the system starts accepting
queries.

> Allow sharing of large in memory data structures across cores
> -------------------------------------------------------------
>
>                 Key: SOLR-8349
>                 URL: https://issues.apache.org/jira/browse/SOLR-8349
>             Project: Solr
>          Issue Type: Improvement
>          Components: Server
>    Affects Versions: 5.3
>            Reporter: Gus Heck
>            Assignee: Noble Paul
>         Attachments: SOLR-8349.patch, SOLR-8349.patch
>
>
> In some cases search components or analysis classes may utilize a large dictionary or
other in-memory structure. When multiple cores are loaded with identical configurations utilizing
this large in memory structure, each core holds it's own copy in memory. This has been noted
in the past and a specific case reported in SOLR-3443. This patch provides a generalized capability,
and if accepted, this capability will then be used to fix SOLR-3443.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message