nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Darren Govoni <dar...@ontrenet.com>
Subject Re: Not Seeing Provenance data
Date Fri, 10 Apr 2020 16:48:42 GMT
It would seem the feature is either broken completely or only works in specific conditions.

Can the Nifi team put a fix on their road map for this?
Its a rather central feature to Nifi.

Sent from my Verizon, Samsung Galaxy smartphone

________________________________
From: Wyllys Ingersoll <wyllys.ingersoll@keepertech.com>
Sent: Friday, April 10, 2020 11:17:42 AM
To: users@nifi.apache.org <users@nifi.apache.org>
Subject: Re: Not Seeing Provenance data

I have a similar problem with viewing provenance.  I have a 3-node cluster in a kubernetes
environment, the provenance_repository directory for each node is on a persistent data store
so it is not deleted or lost between container restarts (which are not very common).  My nifi.provenance.repository.max.storage.time
is 24 hours.

Whenever I try to view any provenance, nothing is ever shown.  If I manually inspect the provenance_repository
directory, there is a lucene index and TOC being created.

I see log messages like these:

Submitting query +processorId:882133fe-b684-148b-ad88-7850437ca591 with identifier 64a703fe-0171-1000-0000-000065abd91a
against index directories [./provenance_repository/lucene-8-index-1560864819888]
Returning the following list of index locations because they were finished being written to
before 1586531601311: []
Found no events in the Provenance Repository. In order to perform maintenace of the indices,
will assume that the first event time is now (1586531601311)


Any suggestions?

-Wyllys Ingersoll



On Thu, Apr 9, 2020 at 11:25 AM Dobbernack, Harald (Key-Work) <harald.dobbernack@key-work.de<mailto:harald.dobbernack@key-work.de>>
wrote:

Hey Mark,



great news and thank you very much!



Happy Holidays!

Harald



Von: Mark Payne <markap14@hotmail.com<mailto:markap14@hotmail.com>>
Gesendet: Donnerstag, 9. April 2020 17:18
An: users@nifi.apache.org<mailto:users@nifi.apache.org>
Betreff: Re: Not Seeing Provenance data



Thanks Harald,



I have created a Jira [1] for this. There’s currently a PR up for it as well.



Thanks

-Mark



[1] https://issues.apache.org/jira/browse/NIFI-7346



On Apr 9, 2020, at 11:14 AM, Dobbernack, Harald (Key-Work) <harald.dobbernack@key-work.de<mailto:harald.dobbernack@key-work.de>>
wrote:



Hi Mark,



I can confirm after testing that if no provenance event has been generated in a time greater
than the set nifi.provenance.repository.max.storage.time then as expected the last recorded
provenance events don’t exist anymore but also from then on any new provenance events are
also not searchable, the provenance Search remains completely empty regardless of how many
flows are active.  As described also *.prov file is then missing in provenance repository.
After restart of Nifi new prov File will be generated and provenance will work again, but
only showing stuff generated since last NiFi Start.



So yes, I’d say your Idea

    ‘If so, then I think that would understand why it deleted the data. It’s trying to
age off old data

     but unfortunately it doesn’t perform a check to first determine whether or not the
“old file”

     that it’s about to delete is also the “active file”.’

fits very nicely to my test.



As a workaround we’re going to set a greater nifi.provenance.repository.max.storage.time
until this can be resolved.



Thanks again for looking into this.

Harald





Von: Dobbernack, Harald (Key-Work)
Gesendet: Donnerstag, 9. April 2020 15:22
An: users@nifi.apache.org<mailto:users@nifi.apache.org>
Betreff: AW: Not Seeing Provenance data



Hi Mark,



thank you for looking into this.



The nifi.provenance.repository.max.storage.time setting might explain why I haven’t been
experiencing the effect so often since changing from the default to 120 hours a few months
ago 😉



But I believe provenance stopped working last time although there was an ‘active’ flows
in wait Processor, expiring every hour, going on to ‘send a message’ before being rerouted
to the same wait processor. I would have expected this generates provenance entries?  As I
am not actually 100% sure if that wait processor was in use when last provenance got lost
I will check with a testing system to see if I can reproduce provenance breakage when no active
flows are around for a time greater  nifi.provenance.repository.max.storage.time and I will
get back to you.



Thank you!

Harald





Von: Mark Payne <markap14@hotmail.com<mailto:markap14@hotmail.com>>
Gesendet: Donnerstag, 9. April 2020 14:41
An: users@nifi.apache.org<mailto:users@nifi.apache.org>
Betreff: Re: Not Seeing Provenance data



Hey Daren, Herald,



Thanks for the note. I have seen this once before but couldn’t figure out what caused it.
Restarting addressed the issue.



I think I may understand the problem, now, though, after looking at it again.



In nifi.properties, there are a couple of property named “nifi.provenance.repository.max.storage.time”
that defaults to “24 hours"

Is it possible that you went 24 hours (or whatever value is set for that property) without
generating any Provenance events?



If so, then I think that would understand why it deleted the data. It’s trying to age off
old data but unfortunately it doesn’t perform a check to first determine whether or not
the “old file” that it’s about to delete is also the “active file”.



Can you confirm whether or not you would expect to see 24 hours pass without any provenance
data?



Thanks

-Mark







On Apr 9, 2020, at 4:32 AM, Dobbernack, Harald (Key-Work) <harald.dobbernack@key-work.de<mailto:harald.dobbernack@key-work.de>>
wrote:



What I noticed is that as long as provenance is working there will be *.prov files in the
directory. When Provenance isn’t working these files are not to be seen. Maybe some Cleaning
Process deletes those files prematurely or the process building them doesn’t work any more?



Von: Dobbernack, Harald (Key-Work) <harald.dobbernack@key-work.de<mailto:harald.dobbernack@key-work.de>>
Gesendet: Donnerstag, 9. April 2020 10:27
An: users@nifi.apache.org<mailto:users@nifi.apache.org>
Betreff: AW: Not Seeing Provenance data



This is something I experience too from time to time. My quick and dirty workaround is stop
nifi, delete everything in the provenance directory, restart….  Then Provenance is usable
again (of course only with data since the delete) . I’m hoping very much there is a better
way, someone can show us better settings or a potential bug can be discovered…



Von: Darren Govoni <darren@ontrenet.com<mailto:darren@ontrenet.com>>
Gesendet: Mittwoch, 8. April 2020 20:31
An: users@nifi.apache.org<mailto:users@nifi.apache.org>
Betreff: Not Seeing Provenance data



Hi,

  When I go to "View data provenance" in Nifi, I never see any logs for my flow. Am I missing
some configuration setting somewhere?



thanks,

Darren





Harald Dobbernack
Key-Work Consulting GmbH | Kriegsstr. 100 | 76133 | Karlsruhe | Germany | https://www.key-work.de
| Datenschutz<https://www.key-work.de/de/footer/datenschutz.html>
Fon: +49-721-78203-264 | E-Mail: harald.dobbernack@key-work.de<mailto:harald.dobbernack@key-work.de>
| Fax: +49-721-78203-10

Key-Work Consulting GmbH, Karlsruhe, HRB 108695, HRG Mannheim
Geschäftsführer: Andreas Stappert, Tobin Wotring


Mime
View raw message