nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koji Kawamura <>
Subject Re: Docker to deploy NiFi
Date Wed, 12 Oct 2016 06:12:02 GMT
Hi Stephane,

I've been using Docker compose to form a NiFi cluster for testing purpose.
It's not on dockerhub, but I've shared it as a Github project. [1]

My knowledge of Docker is fairly limited, but hope this helps.

1) Although I haven't played with those yet, docker-compose has
resource allocation parameters, such as cpu_quota or mem_limit [2]. I
assume these can be helpful to allocate resources in a container pool
using Swarm together.

>From NiFi's perspective,  conf/bootstrap.conf has JVM options. It's
possible to tweak the bootstrap.conf when NiFi starts on a container,
please see the example shell script I used to replace
hostname setting [3]

2) docker-compose has 'scale' command. With that, you can spawn new
NiFi nodes within the same network and get it automatically added to
the cluster, e.g. $ docker-compose scale nifi-nodes=2
I have configured a Zk node, and a NiFi seed node to expose ports, and
nifi-nodes as services in docker-compose.yml [4]

3) Docker container can map local file system to container volume. If
you'd like to preserve NiFi data dirs such as content-repository
across container's life cycle, then I think you can use this feature.



On Tue, Oct 11, 2016 at 3:03 PM, St├ęphane Maarek
<> wrote:
> Hi,
> Has anyone deployed NiFi using docker? I see many images on dockerhub, but I
> have a few questions regarding usage:
> 1) How do you handle overall configuration (java memory options, etc) ?
> 2) If you add a docker process, does the spun up NiFi get automatically
> added to your cluster?
> 3) How do you handle persistence of data ?
> Thanks!
> Stephane

View raw message