nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aldrin Piri <aldrinp...@gmail.com>
Subject Re: MergeContent Processor
Date Thu, 19 Nov 2015 13:43:33 GMT
Elli,

Your understanding of the functionality is correct. There are a couple of
criteria that drive when a bin is "done." In this case, if you establish
the optional maximum properties, these drive in closing out sooner.  That
is if a max age is specified, and any of the bins have gone beyond that
time, they will be closed and transferred out.

Alternatively, a bin is also considered ready if the max age has not yet
elapsed and:

* both minimum size and minimum number of files has been reached and a few
successive attempts to add to the bin (specifically, five) have been
unsuccessful, signaling that it is nearly full or the objects are ill
suited for tighter packing.

* size or number of entries is greater than or equal to their respective,
optionally, specified maximum

Let us know if you have any other questions!

On Thu, Nov 19, 2015 at 8:09 AM, Elli Schwarz <eliezer_schwarz@yahoo.com>
wrote:

> Hello,
>
> I'm a bit confused about the relationship of certain properties of the
> MergeContent processor. Specifically, how do the properties min entries,
> max entries, max bin age, max number of bins interact? If the MergeContent
> processor receives the min number of entries, does it merge without waiting
> for max bin age? Or does max bin age trump the other properties? If max bin
> age is hit before the min number of entries, does the processor wait until
> it gets the min number? Does it merge once it gets to the max bin age,
> regardless of whether or not the max entries has been received? What about
> min/max group size vs. min/max number of entries?
>
> I want to make sure that the processor isn't waiting forever (ie, will
> send after 10 minutes no matter what) if there's only 1 flowfile in the
> queue. If I set max bin age to 10 minutes, and min entries to 10, what does
> that mean, it seems to work the way I expect, which makes me wonder what
> does the min entries property mean if it doesn't seem to be used?
>
> Thank you for any clarifications possible. I looked through the
> documentation for this processor, but it doesn't seem to explain these
> crucial details, which greatly impact my strategy for using this processor
> properly.
>
> -Elli
>
>

Mime
View raw message