nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: NiFi data HA in cluster mode
Date Tue, 09 Jan 2018 06:25:17 GMT
I'd avoid setting any processor to primary node only unless it is a
source processor (something that brings data into the system).

But, yes, I believe your description is accurate as of now.

Thanks

On Mon, Jan 8, 2018 at 11:21 PM, 尹文才 <batman713@gmail.com> wrote:
> Thanks Joe, so you mean for example, if I set one processor to run only on
> primary node in the cluster and there're 100 FlowFiles in the incoming
> queue of the processor
> waiting to be processed by this processor, and the processor suddenly goes
> down and then another node is elected as the primary node, those 100
> FlowFiles will be kept locally
> in the node that went down and will continue to be processed by the node
> when it goes back online, these FlowFiles will not be available to the new
> primary node and other nodes,
> am I correct?
>
> Regards,
> Ben
>
>
> 2018-01-09 14:08 GMT+08:00 Joe Witt <joe.witt@gmail.com>:
>
>> Ben,
>>
>> Data already mid-flow within a node will be kept on the node and
>> processed when the node is back on-line.  All other data coming into
>> the cluster can fail-over to other nodes provided you're sourcing data
>> with queuing semantics or automated load balancing or fail-over as-is
>> present in the Apache NiFi Site to Site protocol.
>>
>> Thanks
>> Joe
>>
>> On Mon, Jan 8, 2018 at 11:05 PM, 尹文才 <batman713@gmail.com> wrote:
>> > Hi guys, I have a question about data HA when NiFi is run in clustered
>> > mode, if one node goes down, will the flowfiles owned by this node taken
>> > over and processed by another node?
>> > Or will the flowfiles be kept locally to that node and will only be
>> > processed when that node is back online? Thanks.
>> >
>> > Regards,
>> > Ben
>>

Mime
View raw message