nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy LoPresto <>
Subject Re: Best practices to replicate a large flow from a standalone NiFi to a cluster
Date Thu, 29 Nov 2018 20:10:34 GMT
The NiFi Registry is the appropriate tool for this scenario. If both the standalone (“dev”)
NiFi instance and the clustered (“prod”) NiFi instance have shared network access, you
can instantiate a single NiFi Registry instance and add a Registry Client pointing to that
Registry to each NiFi instance. The “dev” NiFi can commit a flow to the Registry, and
the “prod” NiFi can instantiate it via “Add Process Group > Import…”. You can
even set event hooks to automate this process if desired. 

If you have your “dev” and “prod” instances separated so that they do not have access
to the same NiFi Registry, you would create multiple Registry instances, and use a tool like
the NiFi CLI to replicate flow definitions from “dev” Registry to “prod” Registry.

I’ve attached a diagram (originally created by Kevin Doran for Hortonworks, I believe) illustrating
a possible deployment scenario. 

Andy LoPresto
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Nov 29, 2018, at 12:03 PM, Erik Anderson <> wrote:
> We are just starting to play with NiFi clusters. I have a question more about migration
of a data flow from a standalone instance of NiFi.
> Lets say, we create an advanced data flow in a stand alone instance of NiFi in a corporate
environment. Corporate environments bring in new challenges, like “StandardProxyConfiguration”,
“StandardRestrictedSSLContext”, Kerberos, HIVEConnectionPool, etc.
> Whats the best way to “replicate” a very complex flow, setup in standalone instances
of NiFi, to a cluster setup using ZooKeeper? I would prefer to use the NiFi Flows Registry.
> Erik Anderson
> Bloomberg

View raw message