nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <mattyb...@apache.org>
Subject Re: Penalize Flow File on Failure
Date Fri, 14 Oct 2016 17:19:27 GMT
Manish,

You could use ExecuteScript with Groovy and the following:

def flowFile = session.get()
if(!flowFile) return
flowFile = session.penalize(flowFile)
session.transfer(flowFile, REL_SUCCESS)

Then you can route "success" back to the processor. You'd set the
Penalty Duration on the ExecuteScript processor configuration
(Settings tab).

Regards,
Matt

On Fri, Oct 14, 2016 at 1:01 PM, Manish Gupta 8 <mgupta50@sapient.com> wrote:
> Thanks Matt. As a workaround, is there a processor that does not modify the flow file
(content or attribute) at all, and I can use it to delay the self-referencing flow files to
hit the main processor again immediately?
>
> Regards,
> Manish
>
> -----Original Message-----
> From: Matt Burgess [mailto:mattyb149@apache.org]
> Sent: Friday, October 14, 2016 10:25 PM
> To: users@nifi.apache.org
> Subject: Re: Penalize Flow File on Failure
>
> Manish,
>
> The use of penalize(), yield(), etc. is not enforced by the framework,
> so processors can have different behavior, sometimes on purpose, and
> sometimes inadvertently.  The Developer's Guide has guidance on when
> to use such methods [1], and reviewers often check the submissions to
> see if they exhibit such behavior, but it is possible for processors
> to handle these cases differently.
>
> Please feel free to log bugs/improvements for such processors, I think
> consistent behavior in this vein (when prudent) is a good idea.
>
> Regards,
> Matt
>
> [1] https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#penalization-vs-yielding
>
> On Fri, Oct 14, 2016 at 12:38 PM, Manish Gupta 8 <mgupta50@sapient.com> wrote:
>> Hello Everyone,
>>
>>
>>
>> In some of the processors I have seen that flow files on failure are not
>> being penalized. For example – Kite processors like ConvertJsonToAvro. Is
>> there some specific reason why some processors have different behavior?
>>
>>
>>
>> I think every processor should penalize every non-success relationship. This
>> way, we can hold those failed files (using a self-referencing loop and some
>> decent penalty duration) in the queue itself and debug and resolve later.
>>
>>
>>
>> Regards,
>>
>> Manish
>>
>>

Mime
View raw message