nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Question about ephemeral NiFi (inside a Docker container) with ZooKeeper
Date Wed, 16 Jan 2019 15:25:31 GMT
Hello,

ZooKeeper is not used at all in stand alone mode.

In a cluster it used for leader election, as well as to store state for
processors that maintain state, such as source processors that keep track
of a time stamp or id.

-Bryan

On Wed, Jan 16, 2019 at 7:49 AM Erik Anderson <eanders@pobox.com> wrote:

> Our NiFi is running in docker container.  I am using some cool ideas from
>
>
> https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/start.sh#L39
>
> Simply set environment variables, stop and restart the container and the
> systems are completely reconfigured.
>
> At the moment, I am using an external volume for the nifi/conf directory.
> Inside this directory is so much contents. Provenance, flow files, canvas,
> NiFi state?
>
> I know if I kill the container and restart it, because of the external
> Docker volume, the system comes backup in its original state. The NiFi
> runtime is ephemeral but due to the external volume, it all restores.
>
> My Question is:
> If I start using Zookeeper, where even the nifi/conf directory becomes
> ephemeral, what is Zookeeper doing for me in both the single instance and
> clustered instances of NiFi.
>
> 1) Single Node NiFi
>   - Does Zookeeper store the provenance events
>   - Does Zookeeper store the state?
>   - Does Zookeeper store the flows and canvas themselves?
>   - Does Zookeeper also store the nifi.properties files and everything
> under nifi/conf?
> 2) Cluster mode -
> https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/start.sh#L34
>   - What is Zookeeper doing in the cluster mode that its not doing in the
> Single Node? Just leader election processes?
>
>
> Thanks,
> Erik Anderson
> Bloomberg
>
-- 
Sent from Gmail Mobile

Mime
View raw message