nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: Configuration Driven Property Updates
Date Thu, 22 Oct 2015 13:08:51 GMT
I two thoughts on this concept. Firstly is the notion of Controller Services. Rather than having
properties that
change significantly depending on the strategy selected, each strategy could be implemented
as a separate
Controller Service. Then, in the Processor you'd just choose which Controller Service to use.
This allows the
properties that need to be configured for that strategy to move into the Controller Service
itself.

The second thought, is one that I had proposed a while back, but it never picked up any traction.
The idea is
to have properties "depend" on other properties. So, for instance, you could say Property
B is only applicable
if Property A is set to Allowable Value 1 or Allowable Value 2. If Property A is not set or
is set to Allowable Value
3, for instance, then Property B is not even shown in the UI.

The nice thing about the second option is that it would make the UI cleaner for existing Processors.
For instance,
we could indicate in CompressContent that the "Compression Level" property should not even
be shown if the
mode is set to Decompress. Currently, we just document that "This property is ignored unless
mode is Compress."
I would much prefer to not even show the property at all.

Any thoughts?

-Mark



> On Oct 21, 2015, at 10:11 PM, Joe Witt <joe.witt@gmail.com> wrote:
> 
> Not to trivialize this but curious if we added an 'apply' type option
> there would that be a sufficient user experience?  The core point is
> once you select a new strategy we need to send that fact to the server
> to see what effect that has.  That happens when you hit ok but as you
> note that closes the dialogue and requires you to reopen it which is
> just sort of awkward.  So would an 'ok' option which applies the
> changes and talks to the server and gets new info but doesn't close
> the dialogue make this good enough?
> 
> Thanks
> Joe
> 
> On Wed, Oct 21, 2015 at 6:09 PM, Michael D. Coon
> <mdcoon1@yahoo.com.invalid> wrote:
>> Mike,
>>  Thanks, that is EXACTLY what I'm doing as well :)    My current version just allows
them to set the properties up onceā€¦if they need to change the selection, they unfortunately
have to create a whole new processor configuration and delete the old one. Kinda stinks but,
as you said, current UI doesn't support this particular use case. At least I'm not alone!
:)
>> Mike
>> 
>> 
>> 
>>     On Wednesday, October 21, 2015 6:02 PM, Michael Moser <moser.mw@gmail.com>
wrote:
>> 
>> 
>> Mike,
>> 
>> I have implemented a NiFi processor very similar to what you describe.  A
>> ServiceLoader discovers the plugins that are available to the processor and
>> populates the allowableValues of a PropertyDescriptor.
>> 
>> In the end I don't think there is a way to resolve your questions/issues in
>> the UI.  After you select a plugin, the user will have to save the
>> processor config, then go back into it to see the new properties that you
>> made available.  And if you change your plugin selection, then the
>> processor will stay invalid until the user deletes the old properties that
>> are no longer needed.  The NiFi framework just doesn't support this edge
>> case.
>> 
>> In the end, for my processor, I just made all properties from all plugins
>> available all the time.  I named the properties to begin with the plugin's
>> name to help users keep track of which plugin they applied to.
>> 
>> -- Mike
>> 
>> 
>> 
>> On Wed, Oct 21, 2015 at 5:50 PM, Michael D. Coon <mdcoon1@yahoo.com.invalid>
>> wrote:
>> 
>>> Hey Tony,
>>>   I do have working code but  not in a public repo. I am, in fact, trying
>>> to use NiFi as a backing engine. I think the bottom line is that I want to
>>> be able to add/remove some configuration properties based on the settings
>>> of other configuration properties. For example, I might specify "ClassA"
>>> for property "PluginClass" and the implementation "ClassA" has its own set
>>> of parameters required to operate. I want to expose those on the UI. It
>>> works as I have it now, but if after setting up "ClassA" as my plugin I
>>> change the "PluginClass" setting to "ClassB", all of ClassA's properties
>>> stick around even though they are no longer what I'm returning from
>>> "getSupportedProperties".  It looks like, at least on the version I'm
>>> using, that NiFi is caching the properties internally and not syncing them
>>> with the properties that I provide through the "getSupportedProperties"
>>> method. More specifically, if I actually set a value on any of the "ClassA"
>>> required properties, they stick around forever. I would think that whatever
>>> I return from the "getSupportedProperties" method should, in fact, be the
>>> set of properties exposed on the UI. But they fall out of sync if I change
>>> my settings and add/remove properties.
>>> Mike
>>> 
>>> 
>>> 
>>>     On Wednesday, October 21, 2015 5:42 PM, Tony Kurc <trkurc@gmail.com>
>>> wrote:
>>> 
>>> 
>>> Mike,
>>> I'm not quite following what you're trying to accomplish. The use case you
>>> described was a bit too meta for me to wrap my head around. It sounds like
>>> a dataflow abstraction layer with nifi as a backing engine. It sounds like
>>> you have some code, is it in a public repo so we could take a look at what
>>> you have?
>>> 
>>> Tony
>>> On Oct 20, 2015 8:58 AM, "Michael D. Coon" <mdcoon1@yahoo.com.invalid>
>>> wrote:
>>> 
>>>> All,
>>>>   I'm building a set of NiFi processors that allow data flow managers to
>>>> dynamically configure reusable data processing modules wrapped inside
>>> NiFi
>>>> processors. Imagine that we might want to use NiFi in some cases and
>>> Storm
>>>> or Spark in others. The business logic in the processing modules would be
>>>> the same regardless of what framework was feeding the data to the
>>>> modules...it allows us to reuse capability in different data flow
>>>> environments.
>>>> 
>>>>   I have this working to some extent using ServiceLoader to discover the
>>>> plugin modules and making those "allowableValues" of a
>>> PropertyDescriptor.
>>>> Now it gets tricky. When the user selects one of the modules, the module
>>>> itself specifies its own set of configuration properties that I'd like to
>>>> expose to the data flow manager on the UI. That too works as I can grab
>>> the
>>>> properties from the module and expose them as NiFi PropertyDescriptors.
>>> So
>>>> far, so good. But when I change my mind and choose a different module
>>> from
>>>> the allowableValues set I created, the old properties remain set with an
>>>> error that they are not supported properties.
>>>> 
>>>>   So how do I get rid of the old properties once a new module has been
>>>> selected? Also, is there any way that the UI can dynamically update the
>>>> required properties once I make a selection of which module I want to
>>> use?
>>>> As it stands, I have to close the settings window and reopen to get the
>>> new
>>>> properties list.
>>>>   Thanks for any advice!
>>>> Mike
>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> 


Mime
View raw message