nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brandon DeVries <...@jhu.edu>
Subject Re: [DISCUSS] Run Once scheduling
Date Thu, 12 Jan 2017 18:41:46 GMT
I think answering Joe's question is step one.  However, I was curious and
tried something:

public void onTrigger(...){
  if(!isSheduled()){
    return;
  }
  FlowFile flowFile = session.get()
  if (flowFile == null){
    return;
  }
  session.transfer(flowFile, REL_SUCCESS);
  updateScheduledFalse();
}

This processes one FlowFile per "scheduling".  I.e., one FlowFile goes
through, and you need to stop / start to get another.  I'm not 100% that
nothing else would ever call the "built in" updateScheduled* methods, but
worst case the processor could maintain its own flag.  Also, for what it's
worth, calling updateScheduledFalse() doesn't "stop" the processor on the
graph... as Oleg mentions, this (or something like this) could potentially
be visually confusing.

I'm not sure how this fits in a production system, but this +
GenerateFlowFile and some backpressure seems possibly useful for
debugging.  I know I've faked this behavior with a GenerateFlowFile w/ run
schedule "1 day" or something before...  then again, maybe it would be best
to not create something that could be confusing / misused in a production
system.

Brandon




On Thu, Jan 12, 2017 at 1:02 PM Joe Witt <joe.witt@gmail.com> wrote:

> Naz,
>
> Why not just leave all the processes running?  If the data only
> arrives periodically that is ok, right?
>
> Thanks
> Joe
>
> On Thu, Jan 12, 2017 at 10:54 AM, Irizarry Jr., Nazario <naz@mitre.org>
> wrote:
> > On a project that I am on we have been looking at using NiFi for
> orchestrations that are invoked infrequently.  For example, once a month a
> new data input product becomes available and then one wants to run it
> through a set of processing steps that can be nicely implemented using NiFi
> processors.  However, using the interval or cron scheduling for this
> purpose begins to get cumbersome after a while with the need to start and
> manually stop these occasional flows.
> >
> > It would be fairly easy to add an additional scheduling option - “Run
> Once” for this use case.  The behavior would be that when a processor is
> set to run once it automatically stops after it has successfully processed
> one input.
> >
> > What do people think?  We are willing to implement this small
> enhancement.
> >
> > Cheers,
> >
> > Naz Irizarry
> > MITRE Corp.
> > 617-893-0074 <(617)%20893-0074>
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message