nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Braddy <rbra...@softnas.com>
Subject RE: Event-driven process scheduling for RSS feed?
Date Sun, 23 Aug 2015 20:25:58 GMT
Thanks Aldin.

I was able to work around the GetFile limitation by using ExecuteStreamCommand with “/bin/cat”
of argument “/pathtofeeds/rssfeed”.  So this flow works:

HandleHttpRequest --> ExecuteStreamCommand (/bin/cat /pathtofeeds/rssfeed) --> HandleHttpResponse

I really like the flexibility to drop into Linux / shell script mode for streaming, which
provides an “out” to do most anything that’s not currently a processor in Nifi.

Best,
Rick

From: Aldrin Piri [mailto:aldrinpiri@gmail.com]
Sent: Sunday, August 23, 2015 3:14 PM
To: users@nifi.apache.org
Subject: Re: Event-driven process scheduling for RSS feed?

Rick,

You would be correct in your analysis of GetFile.  GetFile does not have a processing "mode"
where it is driven by incoming files.

Just to clarify, when you are referencing scheduling, that is approaching "running" of the
processor slightly differently, and dictates when the Flow Controller triggers a processor
to execute.  The two modes are closely related, however.  Some additional details can be located
in the User Guide [1].

Currently, your desired functionality is something that is not currently in a NiFi processor,
and I have created an issue to get it added either as a likely extension of GetFile, as I
can see how it would certainly be helpful. [2]

[1] https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#scheduling-tab
[2] https://issues.apache.org/jira/browse/NIFI-890

On Sun, Aug 23, 2015 at 3:56 PM, Rick Braddy <rbraddy@softnas.com<mailto:rbraddy@softnas.com>>
wrote:
Hi,

I’m trying to understand how to construct an event-driven RSS feed using HandleHttpRequest
and HandleHttpResponse.  I have an XML file as the RSS feed file.

My original inclination was to connect the “success” output from HandleHttpRequest to
trigger a GetFile processor to get the RSS feed file (without deleting it) and send it along
into HandleHttpResponse.  Of course, this failed miserably, because GetFile very quickly filled
up its output queue with 6 GB of RSS feed file copies.  After rewiring things to clear the
queue out, then deleting my queue recovery setup, I went looking at the “Scheduling” tab
of the GetFile processor, hoping to find an “Event-driven” scheduling method; unfortunately,
there is no such thing as truly “event-driven” for a processor (from what I can tell),
and each block runs by default as fast as possible, and can be slowed down to only process
periodically.

I’m guessing there’s a simple solution to my need to create this RSS feed server, but
given I’m still learning Nifi, it’s escaping me at the moment.

Rick


Mime
View raw message