nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Villard <pierre.villard...@gmail.com>
Subject Re: Nifi slowing down the flow in long run
Date Fri, 10 Nov 2017 11:08:14 GMT
Hi,

What implementation are you using for the provenance repository? If you're
using a recent version of NiFi (1.2.0+), you can switch to a new
implementation which is much more performant and should resolve such issues
[1]. The new implementation is
org.apache.nifi.provenance.WriteAheadProvenanceRepository.

[1]
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#provenance-repository

Pierre


2017-11-10 11:57 GMT+01:00 Asanka Sanjaya Herath <angal.sh@gmail.com>:

> ​​
>
> Hi,
>
> I'm using a NiFi instance hosted on google cloud for about  3 months and
> initially, it was running smoothly.
>
> I have mounted *database_repository*, *provinance_repository*,
> *flowfile_repository* and *content_repository* directories to a
> persistent disk because I need the flow files to be persistent whenever the
> VM is redeployed.
>
> I redeployed the VM several times when those directories are being mounted.
>
> Past few days I saw the message "*The rate of the dataflow is exceeding
> the provenance recording rate. slowing down flow to accommodate*." in
> nifi UI.
>
> So that I cleaned up all those mounted directories and redeployed a fresh
> nifi. Now it is running smoothly again.
>
> But cleaning up those directories, in the long run, is going to be a
> problem for us. Any possible solution is available for this? Any help is
> appreciated.
>
> ​
>
> --
>
> *Thanks,*
>
> Asanka Sanjaya Herath
>
> Software Engineer | Zone24x7
>

Mime
View raw message