Mark,

I just read somewhere that the EnforceOrder processor is not meant to be used when numbers will be "skipped". Since im using a timestamp it wont work for me. So I'll need to find another way to do that ordering. I looked at the PriorityAttributePrioritizer, but that doesn't seem to be able to queue for some amount of time before proceeding. Is that correct?

Thanks,
Dan

On Fri, Aug 17, 2018 at 11:13 AM Daniel Watson <dcwatson84@gmail.com> wrote:
Mark,

Thanks! That worked in allowing the DP page to load. Now it exposes the underlying problem, which is that the EnforceOrder processor seems to be adding hundreds of "attributes modified" events, even when no attributes are being modified. Does this have something to do with the flow files in the "wait" state cycling through the processor over and over again? In the examples of that processor I see people looping the "wait" event back around to the processor so that the files are held in that processor until the batch is complete, so that's what I did. Is that an intended usage of it? If so, is there something that can be done about those DP events?

Thanks,
Dan

On Fri, Aug 17, 2018 at 10:30 AM Mark Payne <markap14@hotmail.com> wrote:
OK, my recommendation would be to change to the WriteAheadProvenanceRepository. You can do this by
changing the value of the "nifi.provenance.repository.implementation" property in nifi.properties to 
"org.apache.nifi.provenance.WriteAheadProvenanceRepository"

The WriteAhead implementation was written several releases back, but we left Persistent Provenance Repo
as the default just to ensure that the Write-Ahead implementation was stable enough. At this point, it's pretty
clear that it is not only much faster than the Persistent Prov Repo but also far more stable. There is a JIRA [1]
to make this the new default.

The nice thing is that it is backward compatible, so if you change to the WriteAhead impl and restart, you should
still be able to query all of your existing data without losing anything.

Thanks
-Mark



On Aug 17, 2018, at 10:26 AM, Daniel Watson <dcwatson84@gmail.com> wrote:

Yes.

On Fri, Aug 17, 2018 at 10:19 AM Mark Payne <markap14@hotmail.com> wrote:
OK, thanks. Are you using the default implementation of the Provenance Repository? I.e., the PersistentProvenanceRepository?


On Aug 17, 2018, at 10:10 AM, Daniel Watson <dcwatson84@gmail.com> wrote:

Mark,

1.7.0
06/19/2018 21:55:30 EDT
Tagged nifi-1.7.0-RC1

Thanks

On Fri, Aug 17, 2018 at 10:09 AM Mark Payne <markap14@hotmail.com> wrote:
Hi Daniel,

What version of NiFi are you running?

Thanks
-Mark


> On Aug 17, 2018, at 10:07 AM, Daniel Watson <dcwatson84@gmail.com> wrote:
>
>   Anyone have any issues with the data lineage screen? My NiFi instance can't compute the data lineage for a specific flow. It worked originally, then after running some files through, it no longer will. It gets stuck at 100% and the UI shows nothing.
>
> The flow is...
>
> list files -> update att -> order -> fetch files -> update att -> put files
>
> I removed the provenance directory and restarted, and it did the same thing.