nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carlos Manuel Fernandes (DSI)" <carlos.antonio.fernan...@cgd.pt>
Subject Nifi driven by external configuration
Date Wed, 18 Apr 2018 11:39:06 GMT
Nifi simple principles of flow, attributes of flow ,  processes and  properties of  processors
are very powerful and permit a lot of great combinations.

The problem i see,  is the tendency to repeat flows to make the  same thing, because some
properties are hardwire (properties without expression language)  and doesn't exist a processor
to read External configuration to map to attributes , which can be properties  in processors.

In my Use Cases  I try to address this problem making services with this skeleton :
HandleHttpRequest (receive some Key)  -> GetExternalConfig (Using  Key) -> Others Nifi
Procs with  properties based on attributes read from external config -> HandleHttpResponse
GetExternalConfig is a custom script processor, which transform a sql Query (all the columns
of first row)  in attributes of the current Flow.

In this arrangement , in the zone of "Other Nifi Procs" ,  if properties of processors don't
permit expression language is not possible to use the configuration already read, which is
annoying.
Till now, I had problems with SplitText (Line Split Count) and CSVRecordSetWriter (value separator,
time format, etc), but the problem is general.

If this make sense for more people in the community , we can think to:

1-      Create a Nifi Processor to read external Configuration from relational Databases 
to Attributes. (probably make sense, read from another non relational sources)

2-      By default all the processors properties must admit expression language (probably
some technical ones don't make sense)

If not, I continue  with my custom GetExternalConfig and blaming when I find a property without
expression language :)

Thanks

Carlos
















Mime
View raw message