nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Using [Nifi Flow]-level controllers with NiFi registry flows
Date Thu, 29 Nov 2018 18:50:05 GMT
Dave,

That is true that sensitive properties don't propagate to the registry
and will have to be re-entered upon import, however when the values
are entered it should not trigger a change that needs to be committed,
and whatever is entered locally should be retained across future
upgrades of the versioned process group.

Ed,

I think the challenge is how to correctly capture this information. In
NiFi and NiFi Registry, the  property values are a Map<String,String>
so there would be an entry like "dbcp-connection-pool" ->
"1234-1233-1234-1234", so somewhere in the versioned process group we
would need another map that said "1234-1233-1234-1234" -> "Foo DBCP"
so that we could get the name later during import. I guess every time
a new version is saved you would go through and identify all services
referenced but not included, and then create this map. Not saying it
can't be done, just needs some thought.
On Thu, Nov 29, 2018 at 1:48 PM Pierre Villard
<pierre.villard.fr@gmail.com> wrote:
>
> Hey Dave,
>
> Just want to jump in regarding:
> "Even if the connection properties were identical between environments, sensitive information
does not propagate and the end user would have to re-enter passwords. This will register as
a change in version control and cause problems later when I have a new version of the flow
to import."
>
> I believe that is not correct. It is true that during the initial import of the flow,
the user would have to re-enter passwords. However this will not be considered as a change
in version control and the value won't be changed/removed when upgrading to a new version.
>
> Pierre
>
>
>
> Le jeu. 29 nov. 2018 à 19:38, Ed B <bdesert@gmail.com> a écrit :
>>
>> Bryan,
>> What if we refer controller services not only by UUID, but also by name (at least
in registry).
>> then, during deployment, if matching CS by UUID doesn't exist, we could check all
available CS by a name, and if there is only one matching by type and name CS, we could use
it, otherwise - current functionality should be fine.
>> Thoughts?
>>
>> On Thu, Nov 29, 2018 at 11:55 AM Bryan Bende <bbende@gmail.com> wrote:
>>>
>>> I meant to add that for the case where you are using NiFi's UI to
>>> import the flows, we could consider builder a nice user experience
>>> that prompted the user to select the appropriate services that are
>>> missing, and fill in sensitive properties, etc.
>>>
>>> For the scenario where you are using the CLI, or some scripts, to
>>> import the flows, then we can probably build some kind of convention
>>> into those commands, or add additional commands to help with the
>>> situation.
>>> On Thu, Nov 29, 2018 at 12:51 PM Bryan Bende <bbende@gmail.com> wrote:
>>> >
>>> > Hi Dave,
>>> >
>>> > Currently there isn't a built in way to make this automatic...
>>> >
>>> > The issue is that the versioned flow in registry has the
>>> > PutDatabaseRecord with the DBCP Pool property set to a UUID that only
>>> > existed in the original environment the flow was created in.
>>> >
>>> > When you import the flow to another environment, that UUID is
>>> > obviously not going to exist, but it is also unclear how to select the
>>> > appropriate one. What if there were multiple DBCP connection pools
>>> > visible to where the versioned flow is being imported? There would be
>>> > no way to know which one to use.
>>> >
>>> > I suppose maybe there could be a convention that if there was only one
>>> > matching service of the given type, and it came from the root process
>>> > group, then use that one, but its still hard to know if this is really
>>> > the right service. What if it was for a different database and someone
>>> > didn't realize?
>>> >
>>> > -Bryan
>>> >
>>> > On Thu, Nov 29, 2018 at 12:34 PM David Gallagher
>>> > <dgallagher@cleverdevices.com> wrote:
>>> > >
>>> > > Hi - I'm using nifi-1.7.1 and nifi-registry-0.2.0. I'd like to have
'global' DBCPConnectionPool instances at the Nifi Flow level,  then import flows from the
registry and have them use the global pools, e.g. in a PutDatabaseRecord processor. When I
try that, though, the processor is invalid and the Database Connection Pooling Service shows
'Incompatible Controller Service Configured'. If I manually choose the global controller everything
is fine, but is there a way to have it work so that the matching is automatic?
>>> > >
>>> > >
>>> > > Thanks,
>>> > >
>>> > >
>>> > > Dave

Mime
View raw message