nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jianan Zhang <william.jn.zh...@gmail.com>
Subject Re: About Nar Classloader
Date Fri, 10 May 2019 03:30:53 GMT
Thanks for help.
Very interesting work, it is worth to do a deep research.

Thanks again.
Jianan

On Thu, May 9, 2019 at 10:28 PM Otto Fowler <ottobackwards@gmail.com> wrote:

> I did not update the readme with the bintray maven info, you can see it
> here:
> https://bintray.com/palindromicity
>
>
> On May 9, 2019 at 08:05:08, Otto Fowler (ottobackwards@gmail.com) wrote:
>
> https://www.youtube.com/watch?v=-ISycoP3TVA
> Is a video demonstrating using the archetypes to create a bundle, loading
> it via UI into Metron and starting a storm topology that loads it from HDFS.
>
>
> On May 9, 2019 at 07:47:59, Otto Fowler (ottobackwards@gmail.com) wrote:
>
> I think I am that person, I call them Bundles not NARs. ;)
>
> https://github.com/palindromicity/bundles
> https://github.com/palindromicity/test-bundles
> https://github.com/palindromicity/bundles-testing
> https://github.com/palindromicity/bundles-maven
>
>
> A while ago, I forked and refactored the NAR bundle system  for possible
> use in Apache Metron.  Although we did not end up using it, I created
> github projects for the work.
> While working on this for metron, I used it and archetypes I created for
> metron to use bundles in storm ( loading bundles from HDFS ), and in spring
> boot apps. The system is not up
> to date with the very latest changes in bundles, but for your own app, you
> may not need those changes. My maven plugin and configuration allow you to
> change the names from .nar / NAR to whatever you want.
>
> You are welcome to take a look, as is anyone.  The issues link is there as
> well.
> If you are going to use it, there is a slack that you can join as well (
> though it is for my github org, which also has the simple-syslog* libs that
> Nifi and Metron use ).
>
> # Bundles
>
> Bundles are a derivative of the [Apache Nifi](https://nifi.apache.org)
> [NARs](https://nifi.apache.org/developer-guide.html).
> This is an attempt to adapt the NAR system to be usable outside of [Apache
> Nifi](https://nifi.apache.org) in other scenarios.
>
> ## bundles-maven-plugin
> The [bundles-maven-plugin](https://github.com/palindromicity/bundles-maven)
> is an adapted version of the jar dependency plugin whose function is to
> bundle a jar of jars based on the dependencies for a project.  It also
> creates metadata attributes.
> A project's jar, and it's non-provided dependency jars are place in a /lib
> entry in the bundle, with the bundle itself being in jar format.
>
> ## bundles-lib
> The bundles-lib contains the functionality required to:
> - discover bundles
> - inspect bundles for exposed extension types
> - load the bundles
> - create special class loaders for bundles
> - deliver instances of extension types for use
>
> NAR exposed the bundles through many classes.  I have created the
> BundleSystem interface to expose a more usable, simplified api for our use
> cases.
>
> ### What is different
> - Adaptation of nifi-nar-utils to be used outside of the [Apache Nifi](
> https://nifi.apache.org) project
> - Rudimentary extensibility to allow configuration and injection of
> service types and other things that were hard coded to nifi
> - Refactored from File based to [Apache Commons VFS](
> https://commons.apache.org/proper/commons-vfs/) based, allowing for
> bundle libraries to be loaded from HDFS
> - Rebranding to Bundle from Nar ( although the lib and the plugin allow
> that to be configured now )
> - Added capability to the properties class to write to stream, adapted to
> uri from paths
> - Added integration tests for hdfs
> - Changed to be ClassIndex based instead of ServiceLoader. Service loader
> is slower, and Casey's ClassIndex work is great. This also removes the
> NAR's required manual maintenance of the service file.
> - Refactored to use VFS to load the bundle/nar into the classloader AND to
> use VFS to load the dependency jars -> VFS as a composite filesystem. Thus
> going from NAR's 'working directory', exploded NARS to just loading the
> bundle/nar.
> - Created a simple interface for interacting with the system
> - Refactored such that it is possible to add new bundles to the running
> system without having to restart the application
>
>
> ## Project info
>
> - bundles-lib The bundles library
> - bundles-testing a sample project showing how to use bundles
> - test-bundles source for bundles used for bundles-lib tests
>
>
>
> On May 8, 2019 at 22:53:04, Joe Witt (joe.witt@gmail.com) wrote:
>
> Jianan
>
> We have not done the work to fully isolate the concept of a NiFi Archive
> (Nar) such that it could be used outside of NiFi as a general classloader
> isolation pattern.
>
> There was one other person interested in helping make this happen in the
> past but I'm not sure where it has gone.
>
> With some of the recent advances in Java it might well be the case that a
> more general purpose option exists (modules).
>
> The concept of a NAR is at this time pretty tightly coupled to the NiFi
> project by design.
>
> Thanks
> Joe
>
> On Wed, May 8, 2019 at 10:49 PM Jianan Zhang <william.jn.zhang@gmail.com>
> wrote:
>
>> Hi all,
>>
>> I really like the idea of Nar-Bundle, it create namespaces that separate
>> several dependency environments.
>> I want to use Nar-Bundle in other java project, and I found the
>> <nifi-nar-utils> artifact, but this utils seems to be custom-made to nifi.
>> Does it exists standalone nar-utils or other similar utils?
>>
>> Thanks.
>>
>

Mime
View raw message