nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin Williams <colin.williams.seat...@gmail.com>
Subject Re: Tasks / Time extremely high measure of tasks for LogAttribute
Date Fri, 09 Nov 2018 16:39:03 GMT
Hi Pierre,

Thanks for explaining. Could this cause a performance penalty? I assume
there's no way the next processor can be aware of the penalized flows
state, or this would not be an issue.

On Fri, Nov 9, 2018, 1:36 AM Pierre Villard <pierre.villard.fr@gmail.com
wrote:

> Hi Colin,
>
> That's because when the flow file is routed to failure by the PutS3, the
> flow file is penalized and cannot be processed before a given penalty
> duration by the LogMessage processor. However the LogMessage processor sees
> there is a flow file to process, and then detect that it cannot be
> processed because of the penalty duration. There is a JIRA for that [1]. An
> option could be to set the penalty duration to 0s in the PutS3 processor
> but I don't think that's a great idea as the processor would keep trying
> sending the flow file and it could lead to unwanted pressure on S3.
>
> [1] https://issues.apache.org/jira/browse/NIFI-3229
>
> Pierre
>
>
> Le ven. 9 nov. 2018 à 00:25, Colin Williams <
> colin.williams.seattle@gmail.com> a écrit :
>
>>
>>
>> On Thu, Nov 8, 2018 at 3:24 PM Colin Williams <
>> colin.williams.seattle@gmail.com> wrote:
>>
>>> [image: airflow_tasks_time.png]
>>>
>>>
>>> On Thu, Nov 8, 2018 at 3:23 PM Colin Williams <
>>> colin.williams.seattle@gmail.com> wrote:
>>>
>>>> I have a LogAttribute processor connected to the failure of an
>>>> S3PutObject processor. Then I noticed when the Put fails the
>>>> LogAttribute Tasks/Time shot to 42,045,444 / 00:01:31.664 for In 14
>>>> (109KB) and Out 14 (109KB).
>>>>
>>>> I'm not sure about HTML email or attachments then will attach
>>>> separately. Just curious what this high task value means.
>>>>
>>>

Mime
View raw message