jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Dürig (JIRA) <j...@apache.org>
Subject [jira] [Commented] (OAK-6584) Add tooling API
Date Tue, 14 Nov 2017 16:20:00 GMT

    [ https://issues.apache.org/jira/browse/OAK-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16251644#comment-16251644

Michael Dürig commented on OAK-6584:

Here is an initial implementation of the tool API based on current trunk: https://github.com/mduerig/jackrabbit-oak/commits/OAK-6584.
Please make sure to have a recent version of the [tool API 1.0-SNAPSHOT|https://github.com/mduerig/oak-tooling-api]
in your local Maven cache.

Based on this I was able to also come up with an initial implementation of my Scala based
[script-oak|https://github.com/mduerig/script-oak/commits/OAK-6584] console. 

Loose ends:
* Figure out how to get rid of {{Store.cast()}} while still allowing {{script-oak}} to resolve
{{NodeState}} from {{RecordId}}.
* Review and validate the implementation to avoid accidentally introducing too tight couplings.
I.e. a tool should be able to use the tool API without having to statically depend on a certain
Oak version. I think we are not there yet with this version. 
* Review the naming. {{Store}} is quite general and in conversations we tend to have to further
qualify it in order to make us clear. I think it would be good to choose a name that would
be clear in its own. Same goes for the entities where the simple class name duplicates one
from the segment store (e.g. {{RecordID}}). Implementation wise this deduplication wasn't
too troublesome but I ended up looking up 'which' record id I had in hand various times. 

> Add tooling API
> ---------------
>                 Key: OAK-6584
>                 URL: https://issues.apache.org/jira/browse/OAK-6584
>             Project: Jackrabbit Oak
>          Issue Type: New Feature
>          Components: segment-tar
>            Reporter: Michael Dürig
>            Assignee: Michael Dürig
>              Labels: tooling
>             Fix For: 1.8, 1.7.12
> h3. Current situation
> Current segment store related tools are implemented ad-hoc by potentially relying on
internal implementation details of Oak Segment Tar. This makes those tools less useful, portable,
stable and potentially applicable than they should be.
> h3. Goal
> Provide a common and sufficiently stable Oak Tooling API for implementing segment store
related tools. The API should be independent of Oak and not available for normal production
use of Oak. Specifically it should not be possible to it to implement production features
and production features must not rely on it. It must be possible to implement the Oak Tooling
API in Oak 1.8 and it should be possible for Oak 1.6.
> h3. Typical use cases
> * Query the number of nodes / properties / values in a given path satisfying some criteria
> * Aggregate a certain value on queries like the above
> * Calculate size of the content / size on disk
> * Analyse changes. E.g. how many binaries bigger than a certain threshold were added
/ removed between two given revisions. What is the sum of their sizes?
> * Analyse locality: measure of locality of node states. Incident plots (See https://issues.apache.org/jira/browse/OAK-5655?focusedCommentId=15865973&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15865973).
> * Analyse level of deduplication (e.g. of checkpoint) 
> h3. Validation
> Reimplement [Script Oak|https://github.com/mduerig/script-oak] on top of the tooling
> h3. API draft
> * Whiteboard shot of the [API entities|https://wiki.apache.org/jackrabbit/Oakathon%20August%202017?action=AttachFile&do=view&target=IMG_20170822_163256.jpg]
identified initially.
> * Further [drafting of the API|https://github.com/mduerig/oak-tooling-api] takes place
on Github for now. We'll move to the Apache SVN as soon as considered mature enough and have
a consensus of where to best move it. 

This message was sent by Atlassian JIRA

View raw message