qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robbie Gemmell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (QPID-7666) [linearstore] Enhancements to the write buffer default and per-queue options
Date Wed, 15 Nov 2017 11:51:00 GMT

    [ https://issues.apache.org/jira/browse/QPID-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16253334#comment-16253334

Robbie Gemmell commented on QPID-7666:

[~kpvdr], with the noted revert above, does that mean the configuration options notes in the
previous comment are now incorrect? They don't seem to have been updated since the prior changes
were originally made.

> [linearstore] Enhancements to the write buffer default and per-queue options
> ----------------------------------------------------------------------------
>                 Key: QPID-7666
>                 URL: https://issues.apache.org/jira/browse/QPID-7666
>             Project: Qpid
>          Issue Type: Improvement
>          Components: C++ Broker
>    Affects Versions: qpid-cpp-1.37.0
>            Reporter: Kim van der Riet
>            Assignee: Kim van der Riet
>             Fix For: qpid-cpp-1.37.0
> In applications where large numbers of persistent queues are created on a single broker,
control over the default and per-queue store options are important so that resources are not
wasted or become exhausted prematurely.
> There are two resources of concern:
> # Memory allocated per queue for buffering;
> # Disk resources consumed for journal files.
> Currently, the default setting for memory allocation is 1024kiB/queue (ie 32 pages of
32kiB). The only default setting available is the page size (using the {{wcache-page-size}}
option), and the number of pages is set internally according to the following table:
> ||{{wcache-page-size}}||{{wcache-num-pages}}||total mmap||
> ||(kiB)||(set interanlly)||(kiB)||
> |  4   |  64 |    256 |
> |  8   |  64 |    512|
> | 16   | 32 |    512|
> | 32*  | 32 |   1024|
> | 64   | 16 |   1024|
> |128   |  8 |   1024|
> \* default
> This gives in effect only 3 per-queue memory allocation options: 256kiB, 512kiB and 1024kiB.
> Currently, the values of {{wcache-page-size}} is limited to the powers of 2 between 4
and 128.
> It is suggested that the {{wcache-num-pages}} value be set as as an option rather than
internally, which would give the user much greater control over the per-queue memory footprint
and the buffer geometry (which can be optimized for the disk hardware and queue load characteristics).
> In addition, the upper value limit on {{wcache-page-size}} should be removed (as it is
rather arbitrary) and the default value should be revised.
> Further, it would be valuable to add these parameters to {{qpid-config}} so that if a
queue is created using this tool, the default values for the store parameters can be overridden
on a per-queue basis.
> Disk resource controls are currently in place for the EFP partition ({{efp-partition}})
and the size of the EFP files ({{efp-file-size}}), both as a default and in {{qpid-config}}.
The default values should also be revised.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org

View raw message