jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chetan Mehrotra (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (OAK-3087) [oak-mongo.js] Add utility to cleanup hidden structure under disabled indices
Date Thu, 09 Jul 2015 05:59:04 GMT

    [ https://issues.apache.org/jira/browse/OAK-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14619956#comment-14619956
] 

Chetan Mehrotra edited comment on OAK-3087 at 7/9/15 5:58 AM:
--------------------------------------------------------------

bq. index cleanup - add some methods to assist in removal of hidden structure under disabled
indices

This would be really useful. However can we change the way its invoked
{noformat}
> oak.indexCleanup.findDisabledIndexNodes() //fill initial list of index names to be cleaned
up
> oak.indexCleanup.indexNames //validate index names to be cleaned up
   ["disabledIndex1", "notToBeCleanedDisabledIndex1", "disabledIndex2"]
> oak.indexCleanup.indexNames = ["disabledIndex1", "disabledIndex2"] //update initial list
as required
> oak.indexCleanup.cleanupIndices() //run a dry mode to validate the operations
> oak.indexCleanup.cleanupIndices(true) //carry out actual cleanup
{noformat}

Instead of above if we change it to 
{noformat}
> oak.indexCleanup.cleanupIndices() //It also takes an optional list of index names
> oak.indexCleanup.cleanupIndices(true) //carry out actual cleanup
{noformat}

So the script would ensure that its run in dryMode first (as it does now). But would not require
the user to play with {{oak.indexCleanup}}. Instead the first call itself would tell which
indexes it would be removing

[~mreutegg] Can you review if the approach to validate (more like a guess) whats the value
for {{type}} field is
{noformat}
+        var isIndexDisabled = function(indexName) {
+            var indexDoc = api.findOne("/oak:index/" + indexName);
+            if (indexDoc == null) {
+                return -1;
+            }
+            var type = indexDoc.type;
+
+            if (type == null) {
+                return -2;
+            }
+
+            var maxRev = new Revision("r0-0-0");
+            for (var revStr in type) {
+                var rev = new Revision(revStr);
+                if (rev.isNewerThan(maxRev)) {
+                    maxRev = rev;
+                }
+            }
+
+            var latestTypeValue = type[maxRev.toString()];
+            return ("\"disabled\"" == latestTypeValue);
+        };
{noformat}


was (Author: chetanm):
bq. index cleanup - add some methods to assist in removal of hidden structure under disabled
indices

This would be really useful. However can we change the way its invoked
{noformat}
> oak.indexCleanup.findDisabledIndexNodes() //fill initial list of index names to be cleaned
up
> oak.indexCleanup.indexNames //validate index names to be cleaned up
   ["disabledIndex1", "notToBeCleanedDisabledIndex1", "disabledIndex2"]
> oak.indexCleanup.indexNames = ["disabledIndex1", "disabledIndex2"] //update initial list
as required
> oak.indexCleanup.cleanupIndices() //run a dry mode to validate the operations
> oak.indexCleanup.cleanupIndices(true) //carry out actual cleanup
{noformat}

Instead of above if we change it to 
{noformat}
> oak.indexCleanup.cleanupIndices() //It also takes an optional list of index names
> oak.indexCleanup.cleanupIndices(true) //carry out actual cleanup
{noformat}

So the script would ensure that its run in dryMode first (as it does now). But would not require
the user to play with {{oak.indexCleanup}}. Instead the first call itself would tell which
indexes it would be removing

> [oak-mongo.js] Add utility to cleanup hidden structure under disabled indices
> -----------------------------------------------------------------------------
>
>                 Key: OAK-3087
>                 URL: https://issues.apache.org/jira/browse/OAK-3087
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: mongomk
>            Reporter: Vikas Saurabh
>            Priority: Minor
>         Attachments: 0001-update-removeDescendantsAndSelf-for-upwards-removal.patch,
0002-OAK-3087-Add-some-methods-to-find-disabled-indices-a.patch
>
>
> While disabling property indices, avoids usage of those indices. But, they still maintain
the data already stored under them. That data would keep on consuming storage space without
serving any purpose. Also, it'd pile on mongo's id index.
> While one can delete index definition node to clear these nodes up -- but, it'd be really
slow and a JCR based deleted would first create a HUGE commit while marking all documents
under it as deleted. And, then the actual deletion would happen in next revision GC after
24 hours have past.
> Hence, it might be beneficial to have a low level api in oak-mongo.js, which simply removes
the document from mongo altogether.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message