nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chirag Dewan <chirag.dewa...@yahoo.in>
Subject Re: Polling Processors impact on Latency
Date Tue, 07 Nov 2017 12:02:24 GMT
Thanks Andrew for the quick response.
I am more concerned about the processors polling for flow files on the connection between
the processors?
Thanks,
Chirag

Sent from Yahoo Mail on Android 
 
  On Tue, 7 Nov 2017 at 5:24 PM, Andrew Grande<aperepel@gmail.com> wrote:   
Yes, polling increases latency in some cases. But no, NiFi is not just polling. It has all
kinds of sources, and listening vs polling vs subscribing purely depends on the protocol of
that given processor.

Hope this helps,
Andrew

On Tue, Nov 7, 2017, 1:39 AM Chirag Dewan <chirag.dewan22@yahoo.in> wrote:

Hi All,
I am a layman to NiFi. I am exploring NiFi as a data flow engine to be integrated with my
Flink processing engine. A brief history of our approach : 
We are trying to build a Streaming Data processing engine. We started off with Flink as the
sole core engine, which is responsible for collection(through Flink Sources) as well as processing
the data. 
Soon we fumbled onto NiFi and the data flow world. 
So far, my understanding is that the NiFi processors are poling processors and not Pub-Sub
processors. That makes me wonder, whats the impact of polling on latency? I know I can configure
my processors to tradeoff latency with throughput, but is there a hard set limit on the latency
I can achieve using NiFi? 
As I said, I am layman as yet. Perhaps my understanding is short here. Any leads would be
much appreciated. 
P.S - Not diving much into Event Driven Processors. They look like something which might clear
my thoughts. But since they are marked experimental, would be more interested in understanding
the timer driven processors.
Thanks,
Chirag

  

Mime
View raw message