Hello Edward,
So, if have of to listen to 32,000 tcp connections and I have only 80 cores, and I configure each ListenSyslog instance for 4,000 connections, doesn't each spawn 4,000 threads behind the scene? The tcp connections will be idle most of the time.

thanks
Clay


On Fri, Aug 2, 2019 at 6:10 AM Edward Armes <edward.armes@gmail.com> wrote:
Hi Clay,

Because Nifi underneath uses a thread pool for it's own threading underneath, and each instance processor runs does so in it's own thread, I don't see any reason why not. One thing to note that the way the ListenTCP processor appears to have been written such that it gets all the requests that have been received on that socket and processes them until either it has no more requests left or process or that instance of the processor is no longer scheduled to run.

Hope that helps

Edward

On Fri, Aug 2, 2019 at 11:28 AM Clay Teahouse <clayteahouse@gmail.com> wrote:
Hello All,

I need to listen to and process thousands of persistent TCP connections. I have 10 nodes, each having 8 cores.
My understanding is that with existing NiFi listening processors, such as ListnSyslog, a thread is utilized for each TCP connection. Does this scale? Do I need to write a custom processor that utilizes a thread pool for reading the data from the socket and processing them?

thanks
Clay