nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lee Laim <lee.l...@gmail.com>
Subject Re: Having a processor wait for all inputs
Date Thu, 16 Mar 2017 02:01:52 GMT
Hi Michael,

NiFi offers several ways of accomplishing this.   One way is to use the merge content processor
with a correlation attribute that each of the 3 inputs share. By setting max and minimum flowfiles
to 3, the processor will wait for exactly 3 inputs with a given attribute before combining
them and releasing them to the next processor.
This is assuming that each of the 3 inputs for a given task will only show up once. 

I'm sure others will chime in with different methods but this will get some gears turning.


Have fun!
Lee


> On Mar 15, 2017, at 7:38 PM, Michael John LEWIS <michaellewis@dbs.com> wrote:
> 
> Hi,
> 
> I’ve just started taking a look at Nifi as it seems perfect for a project I’m working
on right now.
> 
> The one thing I’m trying to solve is this – I have a process (say a batch job or
similar) that I want to invoke,
> only once all the inputs have arrived ( a simple dependency graph ).
> 
> i.e. I have 3 files and I don’t want to run a process after each file arrival, but
once all 3 files have arrived.
> 
> Is this easily achievable in Nifi ? Or would I have to think about custom processor?
> 
> Thanks in advance,
> 
> Mike
> 
> 
> CONFIDENTIAL NOTE:
> 
> The information contained in this email is intended only for the use of the individual
or entity named above and may contain information that is privileged, confidential and exempt
from disclosure under applicable law. If the reader of this message is not the intended recipient,
you are hereby notified that any dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this message in error, please immediately notify
the sender and delete the mail. Thank you.

Mime
View raw message