nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: How to wait for multiple processors' flowfile?
Date Thu, 11 May 2017 13:31:40 GMT
Hello,

The Wait processor has a property "Target Signal Count" which can be a
number or expression language, and tells it how many signals to wait for.

-Bryan


On Wed, May 10, 2017 at 9:58 PM, 尹文才 <batman713@gmail.com> wrote:

> Hi Andrew, thanks for the link, I looked into the wait/notify mechanism
> yesterday and found that one wait processor actually could only wait for
> exactly one notify signal, but my case is like this:
> Processor A, B and C, C has to wait for both flow files from A and B
> conditionally,
> if A and B all outputs flow files, C needs to wait for both flow files and
> use both flow files to update data in C;
> if either A or B doesn't emit flowfile, C also needs to use all incoming
> flow files to update data
> I could know in advance in up-stream processors whether A and B would
> produce flowfiles, but I don't know if it's feasible with NIFI to implement
> the following:
> 1. use wait/notify to wait for multiple different signals(do I have to use
> multiple wait processor chained in a flow to do it?)
> 2. use wait/notify to conditionally wait for a signal
>
> Thanks,
>
> Ben
>
> 2017-05-10 11:49 GMT+08:00 Andrew Psaltis <psaltis.andrew@gmail.com>:
>
>> Hi,
>> I think the new Wait / Notify processors, can be used to solve this
>> problem. Koji has a great blog post on it [1].
>>
>>
>> [1] http://ijokarumawak.github.io/nifi/2017/02/02/nifi-notify-batch/
>>
>> Thanks,
>> Andrew
>>
>> On Tue, May 9, 2017 at 11:43 PM, 尹文才 <batman713@gmail.com> wrote:
>>
>>> Hi guys, I'm currently using NiFi to do some ETL work and I need to
>>> handle the following problem:
>>>
>>> for example I have 3 self-defined processor A, B and C.
>>> all of them contain logic to incrementally update a database table. the
>>> table in C is created by table in A join table in B, so C depends on the
>>> flowfiles from A and B.
>>>
>>> So my question is is there a way for C to wait for both flowfiles from A
>>> and B
>>> to arrive and then start the work in C? Thanks.
>>>
>>
>>
>>
>> --
>> Thanks,
>> Andrew
>>
>> Subscribe to my book: Streaming Data <http://manning.com/psaltis>
>> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
>> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
>>
>
>

Mime
View raw message