jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Mueller (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (OAK-9101) Monitoring for maximum number of entries in biggest map record
Date Fri, 05 Jun 2020 13:14:00 GMT

    [ https://issues.apache.org/jira/browse/OAK-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126758#comment-17126758
] 

Thomas Mueller edited comment on OAK-9101 at 6/5/20, 1:13 PM:
--------------------------------------------------------------

I think there is no need to reset the system property within Oak. However, it doesn't need
to be set (and shouldn't be set, for performance reasons) if the max number is below many
million entries. The problem is: 

{noformat}
+                int maxMapRecordSize = Integer.getInteger(MAX_MAP_RECORD_SIZE_KEY, 0);
{noformat}

Compared to the other code in this code path, this is very, very slow. The method is potentially
called very often, and so it shouldn't parse a string if not needed.

It would be much better to move it inside the "if" statement:
{noformat}
                 if (base.size() >= MapRecord.WARN_SIZE) {
+                    int maxMapRecordSize = Integer.getInteger(MAX_MAP_RECORD_SIZE_KEY, 0);
+                    if (base.size() > maxMapRecordSize) {
+                        System.setProperty(MAX_MAP_RECORD_SIZE_KEY, String.valueOf(base.size()));
+                    }
{noformat}

That way, the system property is only read and set very rarely: in case there is a problem.
It's not set in the normal case.

A monitoring system can then (from time to time) check if the system property is set, and
if yes raise an alert. When raising the alert, the monitoring system can also clear the system
property. That way the segment store will set it again if the number is still high (or even
rising). The monitoring system can also see if the number goes up or down. Which is what is
needed. 






was (Author: tmueller):
I think there is no need to reset the system property within Oak. However, it doesn't need
to be set (and shouldn't be set, for performance reasons) if the max number is below many
million entries. The problem is: 

{noformat}
+                int maxMapRecordSize = Integer.getInteger(MAX_MAP_RECORD_SIZE_KEY, 0);
{noformat}

Compared to the other code in this code path, this is very, very slow. The method is potentially
called very often, and so it shouldn't parse a string if not needed.

It would be much better to move it inside the "if" statement:
{noformat}
                 if (base.size() >= MapRecord.WARN_SIZE) {
+                    int maxMapRecordSize = Integer.getInteger(MAX_MAP_RECORD_SIZE_KEY, 0);
+                    if (base.size() > maxMapRecordSize) {
+                        System.setProperty(MAX_MAP_RECORD_SIZE_KEY, String.valueOf(base.size()));
+                    }
{noformat}

That way, the system property is only read and set very rarely: in case there is a problem.
It's not set in the normal case.

A monitoring system can then (from time to time) check if the system property is set, and
if yes raise an alert. When rising the alert, the monitoring system can also clear the system
property. That way the segment store will set it again if the number is still high (or even
rising). The monitoring system can also see if the number goes up or down. Which is what is
needed. 





> Monitoring for maximum number of entries in biggest map record
> --------------------------------------------------------------
>
>                 Key: OAK-9101
>                 URL: https://issues.apache.org/jira/browse/OAK-9101
>             Project: Jackrabbit Oak
>          Issue Type: Task
>          Components: segment-tar
>            Reporter: Andrei Dulceanu
>            Assignee: Andrei Dulceanu
>            Priority: Major
>             Fix For: 1.32.0
>
>         Attachments: OAK-9101.patch
>
>
> With OAK-9095 we introduced logging at different levels when the number of entries in
a map record becomes too large. For monitoring purposes, it would be also nice to have a system
property (i.e. {{oak.segmentNodeStore.maxMapRecordSize}}) which exposes the maximum number
of entries in the biggest map record.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message