nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charlie Meyer <charlie.me...@civitaslearning.com>
Subject Re: Nifi driven by external configuration
Date Wed, 18 Apr 2018 12:50:25 GMT
To solve this problem, I have written code that sets configuration
properties in the variable registry on each nifi instance, then each flow
that is provisioned has its configuration reference those variables. This
way we are able to deploy nifi instances for different environments, etc
but maintain common logic.

On Wed, Apr 18, 2018 at 6:39 AM, Carlos Manuel Fernandes (DSI) <
carlos.antonio.fernandes@cgd.pt> wrote:

> Nifi simple principles of flow, attributes of flow ,  processes and
>  properties of  processors are very powerful and permit a lot of great
> combinations.
>
>
>
> The problem i see,  is the tendency to repeat flows to make the  same
> thing, because some properties are hardwire (properties without expression
> language)  and doesn’t exist a processor to read External configuration to
> map to attributes , which can be properties  in processors.
>
>
>
> In my Use Cases  I try to address this problem making services with this
> skeleton :
>
> HandleHttpRequest (receive some Key)  -> GetExternalConfig (Using  Key) ->
> Others Nifi Procs with  properties based on attributes read from external
> config -> HandleHttpResponse
>
> GetExternalConfig is a custom script processor, which transform a sql
> Query (all the columns of first row)  in attributes of the current Flow.
>
>
>
> In this arrangement , in the zone of “Other Nifi Procs” ,  if properties
> of processors don’t permit expression language is not possible to use the
> configuration already read, which is annoying.
>
> Till now, I had problems with SplitText (Line Split Count*) *and
> CSVRecordSetWriter (value separator, time format, etc), but the problem is
> general.
>
>
>
> If this make sense for more people in the community , we can think to:
>
> 1-      Create a Nifi Processor to read external Configuration from
> relational Databases  to Attributes. (probably make sense, read from
> another non relational sources)
>
> 2-      By default all the processors properties must admit expression
> language (probably some technical ones don’t make sense)
>
>
>
> If not, I continue  with my custom GetExternalConfig and blaming when I
> find a property without expression language J
>
>
>
> Thanks
>
>
>
> Carlos
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Mime
View raw message