nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Variable / Nifi-EL for RPG Endpoint?
Date Thu, 06 Sep 2018 20:46:00 GMT
That makes sense, not having connectivity definitely introduces some
additional work.

The approach you could take is to have a NiFi Registry instance per
environment [1] and then use the NiFi CLI [2] to export and import
between them.

You are right that the RPG and secrets would have to be applied
through some scripts that used the CLI or the REST API and maybe some
conventions you come up with for how you take some config file and
locate the components and which properties need which values, but
anything is possible via the API.

[1] https://pierrevillard.com/2018/04/09/automate-workflow-deployment-in-apache-nifi-with-the-nifi-registry/
[2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli


On Thu, Sep 6, 2018 at 4:33 PM, Jon Logan <jmlogan@buffalo.edu> wrote:
> Yeah, I meant to say NiFi Registry but substituted in Docker instead! Part
> of the reason we've approached NiFi registry with caution is the lack of
> connectivity between installations. Would the recommended approach be
> shipping a registry containing the flows desired, and then shipping the
> NiFi containers without a baked-in flow? Unfortunately that would require a
> manual step step out of the box as the RPG and other potential secrets
> would need to be specified when pulling from the registry into the NiFi
> cluster (though it could probably be done via the API)?
>
> On Thu, Sep 6, 2018 at 4:11 PM, Andy LoPresto <alopresto@apache.org> wrote:
>
>> I think Bryan was referring to Apache NiFi Registry [1] which is the
>> complementary standalone application which hosts version controlled flow
>> snippets.
>>
>> [1] https://nifi.apache.org/registry.html
>>
>> Andy LoPresto
>> alopresto@apache.org
>> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Sep 6, 2018, at 1:09 PM, Jon Logan <jmlogan@buffalo.edu> wrote:
>>
>> Thanks Bryan -- Is that in terms of Docker Registry? We don't currently use
>> Docker Registry, but our plan was to bundle a flow.gz into a Docker image
>> directly. I'm not sure if maybe we're approaching this problem in a weird
>> way? How do most people handle multiple instances in different environments
>> that are disconnected from one another? We considered running a Docker
>> Registry container as an option, or specifying the flow.gz directly as an
>> option...both seem to have some issues and inconveniences.
>>
>> Thanks!
>> Jon
>>
>> On Thu, Sep 6, 2018 at 4:01 PM, Bryan Bende <bbende@gmail.com> wrote:
>>
>> Jon,
>>
>> The RPG URL is treated similar to sensitive properties or parent
>> controller services, meaning that after deploying your flow you can
>> issue an update to the RPG to set the URL to given environment's value
>> and that change will not be considered a change as far as version
>> control and will be retained when upgrading the process group to newer
>> versions.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Thu, Sep 6, 2018 at 3:48 PM, Jon Logan <jmlogan@buffalo.edu> wrote:
>>
>> We are running into a slight issue when we wish to migrate between NiFi
>> cluster instances...the RPG endpoint in our flow seems to disallow
>>
>> setting
>>
>> of a variable, which makes our flow have to be specific to the specific
>> environment it's being deployed in. It seems odd that it won't let us use
>> an expression there and set the value to a variable -- is this an
>> oversight? Is there some design decision here that we're missing?
>>
>> Thanks!
>>
>>
>>
>>

Mime
View raw message