nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: NiFi hangs at Computing Data Lineage... 100% for a specific flow
Date Fri, 17 Aug 2018 14:30:39 GMT
OK, my recommendation would be to change to the WriteAheadProvenanceRepository. You can do
this by
changing the value of the "nifi.provenance.repository.implementation" property in nifi.properties
to
"org.apache.nifi.provenance.WriteAheadProvenanceRepository"

The WriteAhead implementation was written several releases back, but we left Persistent Provenance
Repo
as the default just to ensure that the Write-Ahead implementation was stable enough. At this
point, it's pretty
clear that it is not only much faster than the Persistent Prov Repo but also far more stable.
There is a JIRA [1]
to make this the new default.

The nice thing is that it is backward compatible, so if you change to the WriteAhead impl
and restart, you should
still be able to query all of your existing data without losing anything.

Thanks
-Mark

[1] https://issues.apache.org/jira/browse/NIFI-5482


On Aug 17, 2018, at 10:26 AM, Daniel Watson <dcwatson84@gmail.com<mailto:dcwatson84@gmail.com>>
wrote:

Yes.

On Fri, Aug 17, 2018 at 10:19 AM Mark Payne <markap14@hotmail.com<mailto:markap14@hotmail.com>>
wrote:
OK, thanks. Are you using the default implementation of the Provenance Repository? I.e., the
PersistentProvenanceRepository?


On Aug 17, 2018, at 10:10 AM, Daniel Watson <dcwatson84@gmail.com<mailto:dcwatson84@gmail.com>>
wrote:

Mark,

1.7.0
06/19/2018 21:55:30 EDT
Tagged nifi-1.7.0-RC1

Thanks

On Fri, Aug 17, 2018 at 10:09 AM Mark Payne <markap14@hotmail.com<mailto:markap14@hotmail.com>>
wrote:
Hi Daniel,

What version of NiFi are you running?

Thanks
-Mark


> On Aug 17, 2018, at 10:07 AM, Daniel Watson <dcwatson84@gmail.com<mailto:dcwatson84@gmail.com>>
wrote:
>
>   Anyone have any issues with the data lineage screen? My NiFi instance can't compute
the data lineage for a specific flow. It worked originally, then after running some files
through, it no longer will. It gets stuck at 100% and the UI shows nothing.
>
> The flow is...
>
> list files -> update att -> order -> fetch files -> update att -> put
files
>
> I removed the provenance directory and restarted, and it did the same thing.




Mime
View raw message