metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ottobackwards <>
Subject [GitHub] metron pull request #865: METRON-1212 The bundle System and Maven Plugin (Fe...
Date Tue, 09 Jan 2018 13:15:21 GMT
GitHub user ottobackwards reopened a pull request:

    METRON-1212 The bundle System and Maven Plugin (Feature Branch)

    This PR contains the Bundle system and Maven Plugin.
    The bundle system and the plugin are adapted from the Apache Nifi project.  
    ## bundles-maven-plugin
    The bundles-maven-plugin is an adapted version of the jar dependency plugin whose function
is to bundle a jar of jars based on the dependencies for a project.  It also creates metadata
    A project's jar, and it's non-provided dependency jars are place in a /lib entry in the
bundle, with the bundle itself being in jar format.
    ## bundles-lib 
    The bundles-lib contains the functionality required to:
    - discover bundles
    - inspect bundles for exposed extension types
    - load the bundles
    - create special class loaders for bundles
    - deliver instances of extension types for use
    NAR exposed the bundles through many classes.  I have created the BundleSystem interface
to expose a more usable, simplified api for our use cases.
    ### From the original PR for METRON-777:
    Metron Bundle Plugin
    - adaptation of the nifi plugin
    - more configurable wrt file extension/dependency and metadata naming
    - adaptation of nifi-nar-utils to be used outside of the nifi project
    - rudimentary extensibility to allow configuration and injection of service types and
other things that were hard coded to nifi
    - refactored from File based to VFS based
    - rebranding to Bundle from Nar ( although the lib and the plugin allow that to be configured
now )
    - added capability to the properties class to write to stream, adapted to uri from paths
    - added integration tests for hdfs
    - changed to be ClassIndex based instead of ServiceLoader. Service loader is slower, and
Casey's ClassIndex work is great. This also removes the NAR's required manual maintenance
of the service file.
    - refactored to use VFS to load the bundle/nar into the classloader AND to use VFS to
load the dependency jars -> VFS as a composite filesystem. Thus going from NAR's 'working
directory', exploded NARS to just loading the bundle/nar.
    ## Previous Review
    Please see [@mattf_apache's review](
    ## changes from that review
    I have changed the InitContext operations to have explicit builders, and made it so that
creating a context can be done separately from initialization.  Two contexts can then be 'merged'.
 This is to allow for the addition of new bundles after initialization.
    In preparing this PR I have:
    - made checkstyle fixes
    - fixed several types
    - added a requested set of tests loading and executing simple interface/implementation
from bundle beyond what is already in the bundle-lib tests
    ## Testing
    *` cd bundles-maven-plugin && mvn -q install && cd .. ` must be run once
to install the maven plugin
    * This review is code review and test code review and running only
    * [Test Project]( can be examined
as a simple example of how to create bundles.
    * The has getting started and quickstart sections with some overview of creating
by hand
    ### For all changes:
    - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at
[Metron Jira](

    - [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are
trying to resolve? Pay particular attention to the hyphen "-" character.
    - [x] Has your PR been rebased against the latest commit within the target branch (typically
    - [x] Have you ensured that the full suite of tests and checks have been executed in the
root metron \
    - [x] Have you written or updated unit tests and or integration tests to verify your changes?
    ### For documentation related changes:
    - [x] Have you ensured that format looks appropriate for the output in which it is rendered
by building and verifying the site-book? 

You can merge this pull request into a Git repository by running:

    $ git pull fifth_bundles

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #865
commit 207d367e360074ad53d6f7ece4e1f1febc5a64e1
Author: Otto Fowler <ottobackwards@...>
Date:   2017-09-27T10:29:42Z

    The bundle=lib and maven plugin for bundles.
    as compared to the feature branch the differences are:
    - Added some modules to test loading a bundle, and executing a function
    - checkstyle changes
    - some minor typo fixes

commit 63134bb7af555fe0896152e7f3a8ba5841a8c1e0
Author: Otto Fowler <ottobackwards@...>
Date:   2017-09-27T11:48:50Z

    update readme based on new build options around delay loading

commit 1971262db7f68e9501bf3de1c6b6a10032ace0fc
Author: Otto Fowler <ottobackwards@...>
Date:   2017-10-25T18:43:28Z

    fix versions

commit 4f618979038f4a0ab8685ca22de0d2857df71444
Author: Otto Fowler <ottobackwards@...>
Date:   2017-10-27T16:14:21Z

    expanded the readme with some getting started and quickstart information

commit 98f5e183e1bda7e589772e300094dfa21a436dd9
Author: Otto Fowler <ottobackwards@...>
Date:   2017-10-27T16:17:58Z

    further refinement

commit 773011f65067097cfe3d5750f7397c046621f7cd
Author: Otto Fowler <ottobackwards@...>
Date:   2017-10-27T19:59:44Z

    add some doc around this test project

commit 6a592267295ab2223fd9a628d75f3afb6232fd6f
Author: Otto Fowler <ottobackwards@...>
Date:   2017-10-27T20:04:36Z

    clarify that the code is not built

commit ea802f2ed8822c35931f0a26955837f6c1ba9de9
Author: Otto Fowler <ottobackwards@...>
Date:   2017-10-27T20:07:34Z

    more fixes

commit 1d18bd2582a62fbbf9edc18ae73ec2e1d93e5f88
Author: Otto Fowler <ottobackwards@...>
Date:   2017-11-08T15:25:41Z

    add in the instructions for building metron with the plugin

commit 7c3296a8ab8d0f241d513e7935641368d7dce533
Author: Otto Fowler <ottobackwards@...>
Date:   2017-11-08T15:34:49Z

    plugin build note

commit 4c8363a58841c56512696bbe8b899cd76e899bc7
Author: Otto Fowler <ottobackwards@...>
Date:   2017-11-09T16:10:13Z

    spelling fix, and note on @IndexSubclasses

commit 5a46767cff88999760c82fc1f3fefc3be483e1da
Author: Otto Fowler <ottobackwards@...>
Date:   2017-11-09T16:26:14Z

    note on ClassIndex

commit cd1e574d611c1fe520fcfadfa70244edce4c6e2f
Author: Otto Fowler <ottobackwards@...>
Date:   2017-11-15T16:55:56Z

    Added a usage guide the describes how to create a bundle and use it
    Clean up the bundles-testing

commit 3c1545f1b24d1662bfd7f159ceb419c1c0f67192
Author: Otto Fowler <ottobackwards@...>
Date:   2017-11-15T16:59:31Z

    fix name

commit 567cdeaf4de00cb3ec32b1c5794d958f9f39d894
Author: Otto Fowler <ottobackwards@...>
Date:   2017-11-22T14:15:59Z

    refactored to metron-bundles directory

commit 2bbd3f7244ad77a76ca4984dde2c183b3af96fc5
Author: Otto Fowler <ottobackwards@...>
Date:   2017-11-22T14:52:47Z

    fix parent

commit e4d9ac279c4949cdbbbd6f76c1741f6bdaeb9211
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-12T20:00:32Z

    remove ./ pathing in tests

commit d13b13012fb8badbcb41003ec82c6de844a64624
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-12T20:15:22Z

    change tests to try to track down issue in travis

commit 37587689e9feadeb54465aa7319f114a4e11255b
Author: cstella <cestella@...>
Date:   2017-12-12T20:24:07Z

    METRON-1306: When index template install fails, we should fail the install closes apache/incubator-metron#834

commit 633ccf0a9a73b5ae408cb30fb3a804b301e7dbed
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-12T20:31:51Z

    trying to go to older trusty

commit 6ad350adb5b3b4bf040464da8fab4b0385444102
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-12T20:54:27Z

    revert travis version, do not put stuff into target

commit 4d9262dae01336b477ec7a1c92ad7c3bef0974b4
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-12T21:17:34Z

    try to not use target in these tests

commit 59fec256855ba21b9f4b4981979e340afde06312
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-13T11:53:13Z

    fixup copies

commit 6ca08b9f598b649d60c6b6468164a374b7f6f555
Author: MohanDV <mohan.dv@...>
Date:   2017-12-13T15:55:04Z

    METRON-1343 Swagger UI for User Controller needs request method (MohanDV via ottobackwards)
closes apache/metron#862

commit d9ed1bad1f5b2e2471fbea11353f2947f7f52e13
Author: nickwallen <nick@...>
Date:   2017-12-14T15:59:27Z

    METRON-1349 Full Dev Builds Metron Twice (nickwallen) closes apache/metron#866

commit c4cee6af64eda4db4da3eff86abab7d4ae4ec56a
Author: mmiklavc <michael.miklavcic@...>
Date:   2017-12-14T20:29:49Z

    METRON-1345: Update EC2 README for custom Ansible (mmiklavc via mmiklavc) closes apache/metron#859

commit 7d16af7868e22acacd7cfa44dccc7a5fe053e08c
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-15T13:47:33Z

    Merge remote-tracking branch 'apache/master' into fifth_bundles

commit 558df79961ef9295784fe0cd246aac80e009387d
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-15T14:54:40Z

    format fixes based on review

commit ad3d46b681702d4cf76d1b68d322a1046bd65174
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-15T22:27:27Z

    fix typo per review

commit 306ef833067f8e4dc146ab3f5a98dd78590fb713
Author: Otto Fowler <ottobackwards@...>
Date:   2017-12-30T14:35:22Z

    add headers to markdown files



View raw message