nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Irizarry Jr., Nazario" <...@mitre.org>
Subject Re: [DISCUSS] Run Once scheduling
Date Thu, 12 Jan 2017 19:08:00 GMT
Yes, we found that to keep the UI in sync (make sure it looks stopped after it runs once) the
flow controller's stopProcessor() method has to be called.

Naz Irizarry
MITRE Corp.
617-893-0074



On Jan 12, 2017, at 1:41 PM, Brandon DeVries <brd@jhu.edu<mailto:brd@jhu.edu>>
wrote:

I think answering Joe's question is step one.  However, I was curious and
tried something:

public void onTrigger(...){
 if(!isSheduled()){
   return;
 }
 FlowFile flowFile = session.get()
 if (flowFile == null){
   return;
 }
 session.transfer(flowFile, REL_SUCCESS);
 updateScheduledFalse();
}

This processes one FlowFile per "scheduling".  I.e., one FlowFile goes
through, and you need to stop / start to get another.  I'm not 100% that
nothing else would ever call the "built in" updateScheduled* methods, but
worst case the processor could maintain its own flag.  Also, for what it's
worth, calling updateScheduledFalse() doesn't "stop" the processor on the
graph... as Oleg mentions, this (or something like this) could potentially
be visually confusing.

I'm not sure how this fits in a production system, but this +
GenerateFlowFile and some backpressure seems possibly useful for
debugging.  I know I've faked this behavior with a GenerateFlowFile w/ run
schedule "1 day" or something before...  then again, maybe it would be best
to not create something that could be confusing / misused in a production
system.

Brandon




On Thu, Jan 12, 2017 at 1:02 PM Joe Witt <joe.witt@gmail.com<mailto:joe.witt@gmail.com>>
wrote:

Naz,

Why not just leave all the processes running?  If the data only
arrives periodically that is ok, right?

Thanks
Joe

On Thu, Jan 12, 2017 at 10:54 AM, Irizarry Jr., Nazario <naz@mitre.org<mailto:naz@mitre.org>>
wrote:
On a project that I am on we have been looking at using NiFi for
orchestrations that are invoked infrequently.  For example, once a month a
new data input product becomes available and then one wants to run it
through a set of processing steps that can be nicely implemented using NiFi
processors.  However, using the interval or cron scheduling for this
purpose begins to get cumbersome after a while with the need to start and
manually stop these occasional flows.

It would be fairly easy to add an additional scheduling option - “Run
Once” for this use case.  The behavior would be that when a processor is
set to run once it automatically stops after it has successfully processed
one input.

What do people think?  We are willing to implement this small
enhancement.

Cheers,

Naz Irizarry
MITRE Corp.
617-893-0074 <(617)%20893-0074>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message