nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ed B <bdes...@gmail.com>
Subject Re: Using [Nifi Flow]-level controllers with NiFi registry flows
Date Thu, 29 Nov 2018 20:00:57 GMT
Regardless the current implementation... if this is an acceptable idea, I
can open JIRA for new feature and have separate discussion under that
ticket. Does it makes sense?


On Thu, Nov 29, 2018 at 12:50 PM Bryan Bende <bbende@gmail.com> wrote:

> Dave,
>
> That is true that sensitive properties don't propagate to the registry
> and will have to be re-entered upon import, however when the values
> are entered it should not trigger a change that needs to be committed,
> and whatever is entered locally should be retained across future
> upgrades of the versioned process group.
>
> Ed,
>
> I think the challenge is how to correctly capture this information. In
> NiFi and NiFi Registry, the  property values are a Map<String,String>
> so there would be an entry like "dbcp-connection-pool" ->
> "1234-1233-1234-1234", so somewhere in the versioned process group we
> would need another map that said "1234-1233-1234-1234" -> "Foo DBCP"
> so that we could get the name later during import. I guess every time
> a new version is saved you would go through and identify all services
> referenced but not included, and then create this map. Not saying it
> can't be done, just needs some thought.
> On Thu, Nov 29, 2018 at 1:48 PM Pierre Villard
> <pierre.villard.fr@gmail.com> wrote:
> >
> > Hey Dave,
> >
> > Just want to jump in regarding:
> > "Even if the connection properties were identical between environments,
> sensitive information does not propagate and the end user would have to
> re-enter passwords. This will register as a change in version control and
> cause problems later when I have a new version of the flow to import."
> >
> > I believe that is not correct. It is true that during the initial import
> of the flow, the user would have to re-enter passwords. However this will
> not be considered as a change in version control and the value won't be
> changed/removed when upgrading to a new version.
> >
> > Pierre
> >
> >
> >
> > Le jeu. 29 nov. 2018 à 19:38, Ed B <bdesert@gmail.com> a écrit :
> >>
> >> Bryan,
> >> What if we refer controller services not only by UUID, but also by name
> (at least in registry).
> >> then, during deployment, if matching CS by UUID doesn't exist, we could
> check all available CS by a name, and if there is only one matching by type
> and name CS, we could use it, otherwise - current functionality should be
> fine.
> >> Thoughts?
> >>
> >> On Thu, Nov 29, 2018 at 11:55 AM Bryan Bende <bbende@gmail.com> wrote:
> >>>
> >>> I meant to add that for the case where you are using NiFi's UI to
> >>> import the flows, we could consider builder a nice user experience
> >>> that prompted the user to select the appropriate services that are
> >>> missing, and fill in sensitive properties, etc.
> >>>
> >>> For the scenario where you are using the CLI, or some scripts, to
> >>> import the flows, then we can probably build some kind of convention
> >>> into those commands, or add additional commands to help with the
> >>> situation.
> >>> On Thu, Nov 29, 2018 at 12:51 PM Bryan Bende <bbende@gmail.com> wrote:
> >>> >
> >>> > Hi Dave,
> >>> >
> >>> > Currently there isn't a built in way to make this automatic...
> >>> >
> >>> > The issue is that the versioned flow in registry has the
> >>> > PutDatabaseRecord with the DBCP Pool property set to a UUID that only
> >>> > existed in the original environment the flow was created in.
> >>> >
> >>> > When you import the flow to another environment, that UUID is
> >>> > obviously not going to exist, but it is also unclear how to select
> the
> >>> > appropriate one. What if there were multiple DBCP connection pools
> >>> > visible to where the versioned flow is being imported? There would
be
> >>> > no way to know which one to use.
> >>> >
> >>> > I suppose maybe there could be a convention that if there was only
> one
> >>> > matching service of the given type, and it came from the root process
> >>> > group, then use that one, but its still hard to know if this is
> really
> >>> > the right service. What if it was for a different database and
> someone
> >>> > didn't realize?
> >>> >
> >>> > -Bryan
> >>> >
> >>> > On Thu, Nov 29, 2018 at 12:34 PM David Gallagher
> >>> > <dgallagher@cleverdevices.com> wrote:
> >>> > >
> >>> > > Hi - I'm using nifi-1.7.1 and nifi-registry-0.2.0. I'd like to
> have 'global' DBCPConnectionPool instances at the Nifi Flow level,  then
> import flows from the registry and have them use the global pools, e.g. in
> a PutDatabaseRecord processor. When I try that, though, the processor is
> invalid and the Database Connection Pooling Service shows 'Incompatible
> Controller Service Configured'. If I manually choose the global controller
> everything is fine, but is there a way to have it work so that the matching
> is automatic?
> >>> > >
> >>> > >
> >>> > > Thanks,
> >>> > >
> >>> > >
> >>> > > Dave
>

Mime
View raw message