nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Sequeiros <helloj...@gmail.com>
Subject Re: NiFi stops processing further flowfiles
Date Fri, 24 Nov 2017 16:50:24 GMT
Good afternoon,

My quick guess is you are running out of threads.
If your system can handle it increase the default "max thread" count found:

Left side controller settings > Maximum timer driven thread count

On Fri, Nov 24, 2017 at 10:47 AM Panos Geo <panospanos1983@outlook.com>
wrote:

> Hello all,
>
> We are running NiFi in a docker container that has roughly 500 processors
> running. These include consuming messages (MQTT), interaction with a
> database and some data processing with execute scripts. Our NiFi version is
> v.1.4.0 and this instance has 18GB of RAM with 12GB allocated to Java heap.
>
> Quite recently we increased the load of the incoming messages
> (1000messages/minute) that lead to more files being queued and some
> backpressure. We then have experienced what appears to be some kind of
> deadlock. NiFi stops processing any flowfiles and the processors running
> get stuck (there is a thread running constantly; we see 1 at the top right
> corner of the processor). If we try to stop these processors the icon gets
> changed to stop, but we now have 2 threads stuck at the top right corner of
> the processor. We cannot start these processors again and no other data
> flow is being processed unless we restart NiFi. We have experienced this
> multiple times over the past week.
>
> I am attaching a nifi dump in case it helps. Our nifi.properties is fairly
> standard (we just have smaller retention periods for content and
> provenance).
>
> Any thoughts? What can we do to debug this further?
>
> Many thanks,
> Panos
>
>

Mime
View raw message