nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: [DISCUSS] Feature proposal: "Function Groups" and wormholes
Date Mon, 10 Aug 2015 14:45:00 GMT
Wormhole writeup:
https://cwiki.apache.org/confluence/display/NIFI/Wormhole+Connections

Function Group writeup:
https://cwiki.apache.org/confluence/display/NIFI/Reference-able+Process+Groups

On Mon, Aug 10, 2015 at 10:41 AM, Joe Witt <joe.witt@gmail.com> wrote:
> Mike
>
> "Wormholes sounds functionally similar to JMS Queues with topic subscriptions?"
> Yep - you've got it.  Though we'd still make it easy for folks to
> understand the directionality of the flow because we don't want to
> lose that.
>
>
> Function Groups:  Yeah i can't believe we didn't really come up with
> this before now.  It was inspired by the wormhole concept.  These
> would be crazy useful.
>
>
>
> Thanks
> Joe
>
> On Mon, Aug 10, 2015 at 10:36 AM, Mike Drob <mdrob@apache.org> wrote:
>> Wormholes sounds functionally similar to JMS Queues with topic
>> subscriptions? Function groups are something I've wished for before but
>> didn't think it was possible.
>>
>> On Sun, Aug 9, 2015 at 8:27 AM, Joe Witt <joe.witt@gmail.com> wrote:
>>
>>> Toivo
>>>
>>> I think from your response you did understand it quite well.  I'll try
>>> to turn this and a series of other features/concepts into a series of
>>> wiki pages so it can be more readily consumed, improved, and commented
>>> on.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Sun, Aug 9, 2015 at 8:04 AM, Toivo Adams <toivo.adams@gmail.com> wrote:
>>> > Not sure I understand 'wormhole' concept correctly.
>>> >
>>> > We use a lot of request-response type of processing.
>>> > Processing pipeline contain several processors.
>>> > Processing is done mostly sequentially (and same parts parallel).
>>> > During processing anything can happen. But our goal is to send always
>>> > responds back.
>>> > In case of errors response contains errors data.
>>> >
>>> > There can be different types of problems; some are warnings, some
>>> notices,
>>> > some errors and some fatal errors.
>>> >
>>> > In case of fatal errors we stop processing automatically and jump
>>> directly
>>> > to so called ‘error flow’. (subflow)
>>> > Error flow responsibility is to create response which contains errors and
>>> > send response back to requestor.
>>> > Error flow may also contain some special purpose processors – collect
>>> some
>>> > data for report, alert monitoring, etc.
>>> >
>>> > Now 'wormhole' is very helpful. It’s annoying to add explicit
>>> connections to
>>> > all business processors. Also this reduces readability.
>>> > For us flow should readable and understandable from business point of
>>> view.
>>> > And low level technical behavior should be handled ‘behind the scenes’.
>>> > Ideally business processors should not have ‘redirect to error flow’
>>> output
>>> > (relationship) at all.
>>> >
>>> > At the same time I think 'wormhole' should be used very carefully. Only
>>> when
>>> > really needed.
>>> > Otherwise it may be source of weird bugs and it is hard to follow what is
>>> > going on.
>>> >
>>> >
>>> > Thanks
>>> > toivo
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > View this message in context:
>>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/DISCUSS-Feature-proposal-Function-Groups-and-wormholes-tp2381p2391.html
>>> > Sent from the Apache NiFi (incubating) Developer List mailing list
>>> archive at Nabble.com.
>>>

Mime
View raw message