qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rob Godfrey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (QPID-6766) [Java Broker/Message Stores] Chunk message content in order to optimise memory usage
Date Tue, 01 Mar 2016 15:28:18 GMT

    [ https://issues.apache.org/jira/browse/QPID-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173897#comment-15173897

Rob Godfrey commented on QPID-6766:

* Need to change the store implementations to respect the chunk size that is passed by the
caller, and on retrieval, produce chunks of the same size.

I don't think the last part of the above is necessary/desirable.  Retrieval from the DB should
be in terms of the chunks in the store - for simplicity we may decide to simply reload the
entire message.  I wouldn't expect us to necessarily be managing flow to disk of individual

> [Java Broker/Message Stores] Chunk message content in order to optimise memory usage
> ------------------------------------------------------------------------------------
>                 Key: QPID-6766
>                 URL: https://issues.apache.org/jira/browse/QPID-6766
>             Project: Qpid
>          Issue Type: Improvement
>          Components: Java Broker
>            Reporter: Alex Rudyy
>             Fix For: qpid-java-6.1
> At the moment, we store message content in BDB Message Store in one chunk. On content
saving we create a byte array and copy content bytes from a direct memory into heap in order
to save that on disk. As result, the message content is fully duplicated in direct memory
and heap. Such implementation causes an extensive and inefficient heap usage on storing of
big messages which might cause OOM errors due to heap consumption on message saving. We can
optimize heap memory usage by saving message in chunks with a size equal to Direct Byte Buffer
size which is used to hold content in the direct  memory.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org

View raw message