nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koji Kawamura <>
Subject Re: Apache NiFI - Atlas Bridge Builders Needed
Date Wed, 06 Sep 2017 00:30:28 GMT
Hello Lawrance,

I've been working on NIFI-3709, adding a Apache NiFi reporting task to
report lineage to Apache Atlas.
The original PullRequest did not use Provenance events, but I'm now
adding it so that it can cover more use-cases.

It's still under development and not ready to be reviewed fully, but
you can look at the code to see if it covers your needs, I hope it
does, if not, please let me know.

The core part of analyzing NiFi Provenance events is implemented in
this package, and extensible.
An analyzer is picked by following matching rules:
1. The component type that emitted the provenance event
2. The transit URI of provenance event
3. The event type of the provenance event

For general protocols such as hdfs or hbase can be covered by #2.
(http, jdbc and such can be done similarly in the future)
If specific implementation is needed, it can be done as #1.

The work also has a mechanism to resolve Atlas cluster name from
hostname/ip address.
User can define mapping of those at the ReportingTask's configuration.


On Wed, Sep 6, 2017 at 6:00 AM, Lawrance, Greg J JTST:EX
<> wrote:
> We are working with Apache NiFi and Apache Atlas (within the Hortonworks
> HDP framework) and we are looking for some paid help to develop something
> that meets our needs.  We posted the opportunity as an issue on GitHub
> associated with the repo where code would be delivered
> .
>  Any suggestions of where
> we might advertise / share this information would be greatly appreciated.
> Thanks in advance.
> Greg

View raw message