nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Lim <andrewlim.apa...@gmail.com>
Subject Re: 3 of 7 Expand all Print all In new window SiteToSiteProvenanceReportingTask reports no data when event type filter is defined
Date Thu, 05 Oct 2017 15:02:24 GMT
I had a SiteToSiteProvenanceReportingTask running in 1.3.0, so I gave this a try myself.

I think I’m able to reproduce the issue that Fred is seeing.  But I’m not sure if the
issue is only related to adding an Event Type filter to the reporting task.

With no filter, I generated some DOWNLOAD events by downloading some FlowFiles.  I did see
these come into my NiFi instance that handles the events.  Everything was working as expected.

But then I stopped the reporting task, added the DOWNLOAD filter and then restarted the task.
 Then I downloaded some FlowFiles again.  As Fred was seeing, these events did not come in.

However, then I stopped the reporting task, removed the filter, and then restarted the task.
 Then I generated numerous provenance events (RETRIEVE, DOWNLOAD, etc).  But no events came
in.

So maybe this has something to do with starting/stopping the reporting task in addition to
adding the filter.

Fred, can you file a Jira?  I can add my comments to it.

Thanks,

-Drew


> On Oct 5, 2017, at 9:51 AM, Mark Payne <markap14@hotmail.com> wrote:
> 
> Hi Fred,
> 
> I have used the reporting task with specific event types listed, but I've not run into
issues personally.
> 
> I am curious if you have tried looking for more common event types, such as RECEIVE,
CONTENT_MODIFIED, or ATTRIBUTES_MODIFIED?
> EXPIRE events only fire when FlowFiles get aged off from a Connection due to not being
processed within the
> Connection's "FlowFile Expiration" threshold, and DOWNLOAD events are only triggered
when a user downloads
> the contents of the FlowFile to look at it manually. So both of these are fairly rare
events.
> RECEIVE, ATTRIBUTES_MODIFIED, and CONTENT_MODIFIED, on the other hand, will generally
happen constantly.
> 
> Thanks
> -Mark
> 
> 
> 
>> On Oct 4, 2017, at 4:50 PM, Fred S <fs9387287@gmail.com> wrote:
>> 
>> Dear community.
>> 
>> I'm currently trying to set up a SiteToSiteProvenanceReportingTask to keep track
of DOWNLOAD provenance events.
>> 
>> If I set it up without any filters at all everything gets sent to my dedicated NiFi
instance for handling these messages. When I set a filter in the "Event Type" property (I've
tried "DOWNLOAD" and "DOWNLOAD, EXPIRE") - nothing.
>> 
>> I've been looking around for anyone experiencing a similar issue, without any luck.
Have anyone else set this up successfully without having to log *all* provenance events? Due
to the number of flowfiles I have flying by each day it would not be feasible to go with that
approach.
>> 
>> I see this issue on both NiFi 1.3.0 and 1.4.0. For what it's worth I've also tried
to accomplish this both with and without security/ssl.
>> 
>> If anyone else needs any additional information about my setup, please let me know.
>> 
>> All the best,
>> Fred
> 


Mime
View raw message