lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Created] (LUCENE-5591) ReaderAndUpdates should create a proper IOContext when writing DV updates
Date Thu, 10 Apr 2014 14:42:14 GMT
Shai Erera created LUCENE-5591:
----------------------------------

             Summary: ReaderAndUpdates should create a proper IOContext when writing DV updates
                 Key: LUCENE-5591
                 URL: https://issues.apache.org/jira/browse/LUCENE-5591
             Project: Lucene - Core
          Issue Type: Improvement
            Reporter: Shai Erera


Today we pass IOContext.DEFAULT. If DV updates are used in conjunction w/ NRTCachingDirectory,
it means the latter will attempt to write the entire DV field in its RAMDirectory, which could
lead to OOM.

Would be good if we can build our own FlushInfo, estimating the number of bytes we're about
to write. I didn't see off hand a quick way to guesstimate that - I thought to use the current
DV's sizeInBytes as an approximation, but I don't see a way to get it, not a direct way at
least.

Maybe we can use the size of the in-memory updates to guesstimate that amount? Something like
{{sizeOfInMemUpdates * (maxDoc/numUpdatedDocs)}}? Is it a too wild guess?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message