nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Christianson <achristian...@hortonworks.com>
Subject Re: minifi-cpp config.yml case-sensitivity
Date Thu, 13 Jul 2017 18:05:45 GMT
My idea of renaming the keys shouldn’t require any changes to yaml-cpp. The pseudocode would
be (assuming a simplification that the whole tree is a map of maps, plus leaf values):

normalize(yamlTree):

  if is_leaf(yamlTree):
    ret yamlTree

  newTree = {}

  for key in yamlTree: 
    newKey = to_lower(key)
    newTree[newKey] = normalize(yamlTree[key]) 

  ret newTree

So we would just normalize all key keys in the parsed yaml tree before mapping the yaml config
values to minifi internal config structures.

-Andy

On 7/13/17, 1:53 PM, "Marc" <phrocker@apache.org> wrote:

    Hey Andy,
        The concern I had was not with performance -- as you said, only at load
    and reload time -- but more with maintainability.
    
         If we make the change to the third party directory we'll have the
    concern that we lose this capability when someone copies a new version of
    yaml-cpp into our base directory, perhaps even at review. I'm not a fan of
    maintaining someone else's code.
    
         Can we alleviate this concern by not making the change directly to
    their code in the third party directory?. I briefly entertained the thought
    of using a git submodule and custom branch, but there we would have the
    same issue. My thought is can we do this in such a way that a copy/paste of
    an updated version makes it glaringly obvious that there are changes here.
    I don't think a section of the readme is likely to suffice.
    
    On Thu, Jul 13, 2017 at 1:40 PM, Andy Christianson <
    achristianson@hortonworks.com> wrote:
    
    > On initial investigation, it doesn’t seem trivial to enable support for
    > this in yaml-cpp. It might be feasible to walk the entire parsed yaml tree
    > and map keys to a to_lower or to_upper key. It would come with some
    > computational and possibly memory cost, but only at config load time, which
    > shouldn’t be in the hot path. IMO, the usability improvement would be worth
    > the cost.
    >
    > On 7/13/17, 11:16 AM, "Marc" <phrocker@apache.org> wrote:
    >
    >     Hi Andy,
    >       I believe this makes sense if Yaml-CPP supports the option, but I
    > would
    >     be cautious if we have to change dependencies. If Yaml-CPP does not
    > support
    >     this are you proposing to make the change to our third party directory?
    >
    >     On Thu, Jul 13, 2017 at 11:09 AM, Andy Christianson <
    >     achristianson@hortonworks.com> wrote:
    >
    >     > All,
    >     >
    >     > It appears that the properties in the config.yml are case-sensitive,
    > and
    >     > to add to it, they’re inconsistent. E.g. in the example from the
    > README.md
    >     > file, we have:
    >     >
    >     > …
    >     > Processors:
    >     >     - name: GetFile
    >     >       id: 471deef6-2a6e-4a7d-912a-81cc17e3a206
    >     >       class: org.apache.nifi.processors.standard.GetFile
    >     >       max concurrent tasks: 1
    >     >       scheduling strategy: TIMER_DRIVEN
    >     >       scheduling period: 1 sec
    >     >       penalization period: 30 sec
    >     >       yield period: 1 sec
    >     >       run duration nanos: 0
    >     >       auto-terminated relationships list:
    >     >       Properties:
    >     >           Input Directory: /tmp/getfile
    >     >           Keep Source File: true
    >     > …
    >     >
    >     > The pragmatic course of action would seem to be making the config
    > parser
    >     > case-insensitive, so that users have one less thing to worry about.
    >     >
    >     > If there is no opposition to this, I will enter a Jira ticket to
    > make the
    >     > config parser case-insensitive.
    >     >
    >     > -Andy
    >     >
    >     >
    >
    >
    >
    

Mime
View raw message