nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject [DISCUSS] Feature proposal: "Function Groups" and wormholes
Date Sat, 08 Aug 2015 02:42:38 GMT
Come on with a subject like that people must want to read on...

The model of configuration hierarchy nifi supports with Process
Groups/ports today enforces a very explicit connection from component
or group to another component or group.  This is great in that it is
very intentional and makes it easy for operator to understand
precisely how the data flows.  However, this means that a given
process group cannot be easily reused - you have to make copies of it.

What we propose then is to introduce a concept of a 'Function Group'.
It would be similar in structure to a Process Group yet it would have
a very important difference.  These 'Function Groups' would be
reference-able from anywhere in the flow.  They would have zero or one
input ports and they would have zero or more output ports.  Once a
function group is defined then references to it can be placed anywhere
onto the flow it is needed just like dragging a new processor onto the
flow works.  The framework would automatically retain the context of
the where the objects actually came from in the flow and thus where
they need to come back out.

Similar in concept, but for different use cases then is the idea of a
'wormhole' type input port and output port.  These would be virtual
ports whereby you could refer to that port anywhere on the graph that
you need to and the actual port would be somewhere specific.  In this
sense you would not be required to have explicit connections at all
times.

I am guessing this is a little tough to follow with both my wording
and the lack of visuals.  Will turn this into a wiki page with some
screenshots/concept drawings to help communicate this better.
However, wanted to put this out to start finding if there are any
strong opinions on this concept.

Our current model is explicit but can be too rigid and cause people to
create flows that have way more connections and duplication than are
truly necessary or intended.  By offering these implicit and explicit
models with some UX enhancements I believe we can retain the ability
for an operator to understand and follow how the flow works but reduce
the complexity that can grow in a completely explicit approach as we
have today.

Thanks
Joe

Mime
View raw message