nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jennifer Barnabee <jennifer.barna...@gmail.com>
Subject Re: Configuration Driven Property Updates
Date Thu, 22 Oct 2015 13:16:32 GMT
+1 for the second option.

On Thu, Oct 22, 2015 at 9:08 AM, Mark Payne <markap14@hotmail.com> wrote:

> I two thoughts on this concept. Firstly is the notion of Controller
> Services. Rather than having properties that
> change significantly depending on the strategy selected, each strategy
> could be implemented as a separate
> Controller Service. Then, in the Processor you'd just choose which
> Controller Service to use. This allows the
> properties that need to be configured for that strategy to move into the
> Controller Service itself.
>
> The second thought, is one that I had proposed a while back, but it never
> picked up any traction. The idea is
> to have properties "depend" on other properties. So, for instance, you
> could say Property B is only applicable
> if Property A is set to Allowable Value 1 or Allowable Value 2. If
> Property A is not set or is set to Allowable Value
> 3, for instance, then Property B is not even shown in the UI.
>
> The nice thing about the second option is that it would make the UI
> cleaner for existing Processors. For instance,
> we could indicate in CompressContent that the "Compression Level" property
> should not even be shown if the
> mode is set to Decompress. Currently, we just document that "This property
> is ignored unless mode is Compress."
> I would much prefer to not even show the property at all.
>
> Any thoughts?
>
> -Mark
>
>
>
> > On Oct 21, 2015, at 10:11 PM, Joe Witt <joe.witt@gmail.com> wrote:
> >
> > Not to trivialize this but curious if we added an 'apply' type option
> > there would that be a sufficient user experience?  The core point is
> > once you select a new strategy we need to send that fact to the server
> > to see what effect that has.  That happens when you hit ok but as you
> > note that closes the dialogue and requires you to reopen it which is
> > just sort of awkward.  So would an 'ok' option which applies the
> > changes and talks to the server and gets new info but doesn't close
> > the dialogue make this good enough?
> >
> > Thanks
> > Joe
> >
> > On Wed, Oct 21, 2015 at 6:09 PM, Michael D. Coon
> > <mdcoon1@yahoo.com.invalid> wrote:
> >> Mike,
> >>  Thanks, that is EXACTLY what I'm doing as well :)    My current
> version just allows them to set the properties up once…if they need to
> change the selection, they unfortunately have to create a whole new
> processor configuration and delete the old one. Kinda stinks but, as you
> said, current UI doesn't support this particular use case. At least I'm not
> alone! :)
> >> Mike
> >>
> >>
> >>
> >>     On Wednesday, October 21, 2015 6:02 PM, Michael Moser <
> moser.mw@gmail.com> wrote:
> >>
> >>
> >> Mike,
> >>
> >> I have implemented a NiFi processor very similar to what you describe.
> A
> >> ServiceLoader discovers the plugins that are available to the processor
> and
> >> populates the allowableValues of a PropertyDescriptor.
> >>
> >> In the end I don't think there is a way to resolve your
> questions/issues in
> >> the UI.  After you select a plugin, the user will have to save the
> >> processor config, then go back into it to see the new properties that
> you
> >> made available.  And if you change your plugin selection, then the
> >> processor will stay invalid until the user deletes the old properties
> that
> >> are no longer needed.  The NiFi framework just doesn't support this edge
> >> case.
> >>
> >> In the end, for my processor, I just made all properties from all
> plugins
> >> available all the time.  I named the properties to begin with the
> plugin's
> >> name to help users keep track of which plugin they applied to.
> >>
> >> -- Mike
> >>
> >>
> >>
> >> On Wed, Oct 21, 2015 at 5:50 PM, Michael D. Coon
> <mdcoon1@yahoo.com.invalid>
> >> wrote:
> >>
> >>> Hey Tony,
> >>>   I do have working code but  not in a public repo. I am, in fact,
> trying
> >>> to use NiFi as a backing engine. I think the bottom line is that I
> want to
> >>> be able to add/remove some configuration properties based on the
> settings
> >>> of other configuration properties. For example, I might specify
> "ClassA"
> >>> for property "PluginClass" and the implementation "ClassA" has its own
> set
> >>> of parameters required to operate. I want to expose those on the UI. It
> >>> works as I have it now, but if after setting up "ClassA" as my plugin I
> >>> change the "PluginClass" setting to "ClassB", all of ClassA's
> properties
> >>> stick around even though they are no longer what I'm returning from
> >>> "getSupportedProperties".  It looks like, at least on the version I'm
> >>> using, that NiFi is caching the properties internally and not syncing
> them
> >>> with the properties that I provide through the "getSupportedProperties"
> >>> method. More specifically, if I actually set a value on any of the
> "ClassA"
> >>> required properties, they stick around forever. I would think that
> whatever
> >>> I return from the "getSupportedProperties" method should, in fact, be
> the
> >>> set of properties exposed on the UI. But they fall out of sync if I
> change
> >>> my settings and add/remove properties.
> >>> Mike
> >>>
> >>>
> >>>
> >>>     On Wednesday, October 21, 2015 5:42 PM, Tony Kurc <
> trkurc@gmail.com>
> >>> wrote:
> >>>
> >>>
> >>> Mike,
> >>> I'm not quite following what you're trying to accomplish. The use case
> you
> >>> described was a bit too meta for me to wrap my head around. It sounds
> like
> >>> a dataflow abstraction layer with nifi as a backing engine. It sounds
> like
> >>> you have some code, is it in a public repo so we could take a look at
> what
> >>> you have?
> >>>
> >>> Tony
> >>> On Oct 20, 2015 8:58 AM, "Michael D. Coon" <mdcoon1@yahoo.com.invalid>
> >>> wrote:
> >>>
> >>>> All,
> >>>>   I'm building a set of NiFi processors that allow data flow managers
> to
> >>>> dynamically configure reusable data processing modules wrapped inside
> >>> NiFi
> >>>> processors. Imagine that we might want to use NiFi in some cases and
> >>> Storm
> >>>> or Spark in others. The business logic in the processing modules
> would be
> >>>> the same regardless of what framework was feeding the data to the
> >>>> modules...it allows us to reuse capability in different data flow
> >>>> environments.
> >>>>
> >>>>   I have this working to some extent using ServiceLoader to discover
> the
> >>>> plugin modules and making those "allowableValues" of a
> >>> PropertyDescriptor.
> >>>> Now it gets tricky. When the user selects one of the modules, the
> module
> >>>> itself specifies its own set of configuration properties that I'd
> like to
> >>>> expose to the data flow manager on the UI. That too works as I can
> grab
> >>> the
> >>>> properties from the module and expose them as NiFi
> PropertyDescriptors.
> >>> So
> >>>> far, so good. But when I change my mind and choose a different module
> >>> from
> >>>> the allowableValues set I created, the old properties remain set with
> an
> >>>> error that they are not supported properties.
> >>>>
> >>>>   So how do I get rid of the old properties once a new module has been
> >>>> selected? Also, is there any way that the UI can dynamically update
> the
> >>>> required properties once I make a selection of which module I want to
> >>> use?
> >>>> As it stands, I have to close the settings window and reopen to get
> the
> >>> new
> >>>> properties list.
> >>>>   Thanks for any advice!
> >>>> Mike
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message