nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charlie Meyer <charlie.me...@civitaslearning.com>
Subject NiFi Remote Process Group Performance Tuning
Date Tue, 20 Nov 2018 03:13:47 GMT
Hi,

I have a setup with 2 nifi deployments that communicate via HTTP remote
process groups.

I have noticed that often, flowfiles will backpressure buffer on the sender
side of the RPG.  Here is an example of what I see:

[image: image.png]

The incoming connection to the RPG is configured with a limit of 500k flow
files in this example. I only see 1 "thread" running for the sender RPG.
Looking at the receiving input port that this RPG is communicating with, I
have it configured like:

[image: image.png]
I have bumped up the number of threads allocated to the input port to 4,
but yet the connection out of the input port to the rest of the flow is not
backing up.

Are there things that can be tuned or other RPG best practices that can be
applied to the sender side of the RPG to increase throughput?

Thanks!

-Charlie

Mime
View raw message