nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <>
Subject Re: NiFI Custom Plugin - Yield Duration Ignored
Date Tue, 17 Nov 2015 16:46:33 GMT
Perhaps we may need to start explaining ‘administrative yield” as Circuit Breaker because
that ’s what it is from known pattern perspective

Question to DEV; Should we consider going as far as deprecating the name and incorporating
CB in it somehow? That would definitely help to avoid the confusion

> On Nov 17, 2015, at 11:19 AM, Mark Payne <> wrote:
> Michael,
> Hello. First, thanks for your interest in NiFi! I'm very happy to see that you're already
starting to dig in pretty deep here!
> I admit that the wording here can be a little confusing. Let me try to explain what's
going on here and how yielding
> works. When you are developing a Processor in NiFi, especially if reaching out to some
external resource,
> a common occurrence is to realize that there is no work that can really be done at this
point in time (for
> example, because the remote resource is not accessible). In that case, the processor
can call ProcessContext.yield()
> and as a result, the framework will stop scheduling that processor to run for the configured
amount of time, so that
> we aren't wasting resources when we know there is nothing that we can accomplish. This
is where the configured
> Yield Duration comes in.
> There is, however, a concept of "administratively yielding" a processor. The idea here
is that the Processor throws
> some unexpected exception. Since the Exception was not of type ProcessException, the
framework considers this
> a bug and will perform an administrative yield to avoid continually triggering a processor
that may be in a bad state.
> Perhaps using the term "administratively yielding" was a bad idea, as it can clearly
lead to confusion.
> If you feel that something is not behaving properly, though, or you believe that the
user experience can be improved
> in some way, please do let us know, as we strive to continually improve the user experience.
> Thanks
> -Mark
>> On Nov 17, 2015, at 10:34 AM, Michael De Courci <> wrote:
>> FYI
>>> Begin forwarded message:
>>> From: Michael De Courci <>
>>> Date: 16 November 2015 at 15:41:27 GMT
>>> To:,, Kefah Issa <>
>>> Subject: NiFI Custom Plugin - Yield Duration Ignored
>>> I wrote a NAR custom plugin to verify NoFi Exception handling. - version; nifi-0.3.0
>>> The plugin was written to always through a RunTimeException for any exception
including ProcessException.
>>> I created a simple flow; GetFile->MyPlugin->PutFile 
>>> and set the Yield duration to 60 seconds.
>>> The log shows that the yield duration is always 1 sec
>>> I debugged the NiFi framework code and see that the yield duration always defaults
to 1 sec;
>>> See;
>>> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent
>>> Setting the duration updates the class 
>>> org.apache.nifi.controller.StandardProcessorNode
>>> but this does not communicate the change to TimerDrivenSchedulingAgent
>>> Evidence from log;
>>> 80eb-552a-471c-8ffd-f2509d71fd2e] failed to process session due to java.lang.RuntimeException:
>>> 2015-11-15 16:17:58,417 WARN [Timer-Driven Process Thread-7] c.b.n.p.SampleErrorHandlerProcessor
SampleErrorHandlerProcessor[id=8e8f80eb-552a-471c-8ffd-f2509d71fd2e] Processor Administratively
Yielded for 1 sec due to processing failure
>>> 2015-11-15 16:17:58,417 WARN [Timer-Driven Process Thread-7] o.a.n.c.t.ContinuallyRunProcessorTask
Administratively Yielding SampleErrorHandlerProcessor[id=8e8f80eb-552a-471c-8ffd-f2509d71fd2e]
due to uncaught Exception: java.lang.RuntimeException
>>> 2015-11-15 16:17:58,419 WARN [Timer-Driven Process Thread-7] o.a.n.c.t.ContinuallyRunProcessorTask
>>> java.lang.RuntimeException: null
>>>       at
>>> With Kind Regards
>>> Michael de Courci

View raw message