nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ed B <bdes...@gmail.com>
Subject Re: Using [Nifi Flow]-level controllers with NiFi registry flows
Date Fri, 30 Nov 2018 05:19:38 GMT
Bryan,
I couldn't find existing JIRA for such feature. So I opened new one. Would
appreciate if you could review.
https://issues.apache.org/jira/browse/NIFI-5856

On Thu, Nov 29, 2018 at 2:20 PM Bryan Bende <bbende@gmail.com> wrote:

> David, yes putting the DB URL into the variable registry should work.
>
> Ed, I'm ok with opening a JIRA for that, I feel like it may have come
> up before, but can't remember if there is already a JIRA.
> On Thu, Nov 29, 2018 at 3:13 PM David Gallagher
> <dgallagher@cleverdevices.com> wrote:
> >
> > @Bryan Bende, @Pierre, you are correct! I'm seeing a version change
> because I changed the database URL, not due to the passwords. Hopefully I
> can move those settings to the database registry instead.
> > ________________________________
> > From: Bryan Bende <bbende@gmail.com>
> > Sent: Thursday, November 29, 2018 1:50 PM
> > To: users@nifi.apache.org
> > Subject: Re: Using [Nifi Flow]-level controllers with NiFi registry flows
> >
> > Dave,
> >
> > That is true that sensitive properties don't propagate to the registry
> > and will have to be re-entered upon import, however when the values
> > are entered it should not trigger a change that needs to be committed,
> > and whatever is entered locally should be retained across future
> > upgrades of the versioned process group.
> >
> > Ed,
> >
> > I think the challenge is how to correctly capture this information. In
> > NiFi and NiFi Registry, the  property values are a Map<String,String>
> > so there would be an entry like "dbcp-connection-pool" ->
> > "1234-1233-1234-1234", so somewhere in the versioned process group we
> > would need another map that said "1234-1233-1234-1234" -> "Foo DBCP"
> > so that we could get the name later during import. I guess every time
> > a new version is saved you would go through and identify all services
> > referenced but not included, and then create this map. Not saying it
> > can't be done, just needs some thought.
> > On Thu, Nov 29, 2018 at 1:48 PM Pierre Villard
> > <pierre.villard.fr@gmail.com> wrote:
> > >
> > > Hey Dave,
> > >
> > > Just want to jump in regarding:
> > > "Even if the connection properties were identical between
> environments, sensitive information does not propagate and the end user
> would have to re-enter passwords. This will register as a change in version
> control and cause problems later when I have a new version of the flow to
> import."
> > >
> > > I believe that is not correct. It is true that during the initial
> import of the flow, the user would have to re-enter passwords. However this
> will not be considered as a change in version control and the value won't
> be changed/removed when upgrading to a new version.
> > >
> > > Pierre
> > >
> > >
> > >
> > > Le jeu. 29 nov. 2018 à 19:38, Ed B <bdesert@gmail.com> a écrit :
> > >>
> > >> Bryan,
> > >> What if we refer controller services not only by UUID, but also by
> name (at least in registry).
> > >> then, during deployment, if matching CS by UUID doesn't exist, we
> could check all available CS by a name, and if there is only one matching
> by type and name CS, we could use it, otherwise - current functionality
> should be fine.
> > >> Thoughts?
> > >>
> > >> On Thu, Nov 29, 2018 at 11:55 AM Bryan Bende <bbende@gmail.com>
> wrote:
> > >>>
> > >>> I meant to add that for the case where you are using NiFi's UI to
> > >>> import the flows, we could consider builder a nice user experience
> > >>> that prompted the user to select the appropriate services that are
> > >>> missing, and fill in sensitive properties, etc.
> > >>>
> > >>> For the scenario where you are using the CLI, or some scripts, to
> > >>> import the flows, then we can probably build some kind of convention
> > >>> into those commands, or add additional commands to help with the
> > >>> situation.
> > >>> On Thu, Nov 29, 2018 at 12:51 PM Bryan Bende <bbende@gmail.com>
> wrote:
> > >>> >
> > >>> > Hi Dave,
> > >>> >
> > >>> > Currently there isn't a built in way to make this automatic...
> > >>> >
> > >>> > The issue is that the versioned flow in registry has the
> > >>> > PutDatabaseRecord with the DBCP Pool property set to a UUID that
> only
> > >>> > existed in the original environment the flow was created in.
> > >>> >
> > >>> > When you import the flow to another environment, that UUID is
> > >>> > obviously not going to exist, but it is also unclear how to select
> the
> > >>> > appropriate one. What if there were multiple DBCP connection pools
> > >>> > visible to where the versioned flow is being imported? There would
> be
> > >>> > no way to know which one to use.
> > >>> >
> > >>> > I suppose maybe there could be a convention that if there was
only
> one
> > >>> > matching service of the given type, and it came from the root
> process
> > >>> > group, then use that one, but its still hard to know if this is
> really
> > >>> > the right service. What if it was for a different database and
> someone
> > >>> > didn't realize?
> > >>> >
> > >>> > -Bryan
> > >>> >
> > >>> > On Thu, Nov 29, 2018 at 12:34 PM David Gallagher
> > >>> > <dgallagher@cleverdevices.com> wrote:
> > >>> > >
> > >>> > > Hi - I'm using nifi-1.7.1 and nifi-registry-0.2.0. I'd like
to
> have 'global' DBCPConnectionPool instances at the Nifi Flow level,  then
> import flows from the registry and have them use the global pools, e.g. in
> a PutDatabaseRecord processor. When I try that, though, the processor is
> invalid and the Database Connection Pooling Service shows 'Incompatible
> Controller Service Configured'. If I manually choose the global controller
> everything is fine, but is there a way to have it work so that the matching
> is automatic?
> > >>> > >
> > >>> > >
> > >>> > > Thanks,
> > >>> > >
> > >>> > >
> > >>> > > Dave
>

Mime
View raw message