nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: What will happen if we lost a node in cluster?
Date Wed, 15 May 2019 15:37:30 GMT
Hello,

Each node of the cluster is running a copy of the dataflow that you
see in the UI, and the data that is being operated on is what is
divided across the nodes (depending on how the data enters the flow).

If a node goes down, the other nodes continue processing, and the data
that is on the node that went down is stuck there until the node is
brought back up.

If the repositories are on mountable drives then you could spin up
another node and re-attach those volumes to pick up where the node
was, in addition you can use standard RAID configurations to avoid the
most common cause of disk failures.

There is also a new effort in the community called "stateless nifi"
(formerly nifi-fn), which would be an alternative approach and would
be able to handle failures differently when the source of data offers
some type of replayable acknowledgment protocol (i.e. something like
kafka).

Thanks,

Bryan

On Tue, May 14, 2019 at 11:17 PM Jianan Zhang
<william.jn.zhang@gmail.com> wrote:
>
> Hi all,
>
> We have a nifi cluster with 3 node. If we lost a node, what will happen to the job running
on the disconnected node.
> The cluster will do nothing till the node is recovered, or cluster will reallocate the
job running on the dead node to other healthy node.
>
> Thanks
> Jianan

Mime
View raw message