uima-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marshall Schor (JIRA)" <...@uima.apache.org>
Subject [jira] [Resolved] (UIMA-4281) Gradually shrink internal core arrays on reset actions
Date Mon, 04 May 2015 20:09:07 GMT

     [ https://issues.apache.org/jira/browse/UIMA-4281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Marshall Schor resolved UIMA-4281.
----------------------------------
    Resolution: Fixed

> Gradually shrink internal core arrays on reset actions
> ------------------------------------------------------
>
>                 Key: UIMA-4281
>                 URL: https://issues.apache.org/jira/browse/UIMA-4281
>             Project: UIMA
>          Issue Type: Improvement
>          Components: Core Java Framework
>            Reporter: Marshall Schor
>            Assignee: Marshall Schor
>            Priority: Minor
>             Fix For: 2.7.1SDK
>
>
> Many internal core data structures composed of arrays (typically of ints) expand as needed
while running.  When a reset event occurs (e.g., cas.reset()), many of these are cleared,
but not "shrunk".  So the effect of running, say, one Document which is very large might cause
many internal data structures to expand, and from then on consume potentially large but unused
space.  
> This isn't too bad an issue with modern OSs; except for movement in the heap, it would
be expected that the pages containing the unused space would gradually be paged out.
> Nevertheless, it would be good to gradually shrink these spaces back down after some
peak.  The shrinkage should be gradual, to avoid needing to expand the arrays too often. 
I think a good rule of thumb would be to remember one previous size, and to shrink only if
both previous sizes were small enough to be contained in 1 size shrinkage.
> This should reduce oscillating back and forth around a particular size, while recovering
space over many "resets" from an early abnormal expansion.
> I'm thinking that the 1 size shrinkage algorithm should run the expansion algorithm backwards
- that is, in many of our data structures, expand by doubling up to some switch point, and
then expand linearly by adding. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message