nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc <phroc...@apache.org>
Subject Re: MiNiFi C++ Expression Language
Date Thu, 04 May 2017 15:25:51 GMT
Andy,
    Depending on the type of parser chosen you may run into memory
limitations on devices that run MiNiFi-C++. Since that's a deployment
concern -- and one that few will face, I would add that we can make this a
conditional build and only including the necessary components within the
binary if a build option is not specified to exclude these components.

On Thu, May 4, 2017 at 11:14 AM, Andrew Christianson <
andrew.christianson@nextcentury.com> wrote:

> The island grammar sounds very appealing for an MVP. Simple to implement,
> yet it covers a very common EL use case (dynamically inserting attr vals
> into property values). If we have general consensus I would love to see a
> MINIFI JIRA ticket added for this.
>
> -Andy
> ________________________________________
> From: Matt Burgess <mattyb149@gmail.com>
> Sent: Thursday, May 4, 2017 11:09:46 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> What you have sounds good to me. IMO minimum viable product would be an
> island grammar (meaning you can have any characters outside a ${}
> expression) and inside would support an attribute name. Next steps could be
> nested expressions and/or support for functions, added piecemeal as the
> contributor sees fit. I wasn't involved in the early development of the EL
> grammar, so maybe someone who was has some thoughts on a natural evolution.
>
> Regards,
> Matt
>
> > On May 4, 2017, at 11:00 AM, Andrew Christianson <andrew.christianson@
> nextcentury.com> wrote:
> >
> > My bad, what does the sketch of the plan *look like*?
> >
> > -Andy
> > ________________________________________
> > From: Andrew Christianson
> > Sent: Thursday, May 4, 2017 10:59:07 AM
> > To: dev@nifi.apache.org
> > Subject: Re: MiNiFi C++ Expression Language
> >
> > What does the sketch of the plan to do the separate implementation?
> Write a flex/bison grammar, hook it into the cmake build, and start using
> it? Any constraints on features or syntax that this separate implementation
> must support?
> >
> > -Andy
> > ________________________________________
> > From: Matt Burgess <mattyb149@gmail.com>
> > Sent: Thursday, May 4, 2017 9:37:34 AM
> > To: dev@nifi.apache.org
> > Subject: Re: MiNiFi C++ Expression Language
> >
> > No plans that I know of. In the meantime, EL support for MiNiFi is kind
> of held hostage, so maybe the separate implementation is more viable in the
> nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever
> exists by then with the cross-platform ANTLR target generation, and test
> the whole kit and caboodle.
> >
> > Anyone out there familiar with the aforementioned tools (or willing to
> hand-roll one)? What do you think about this "EL bootstrapping" approach?
> >
> > Thanks,
> > Matt
> >
> >> On May 4, 2017, at 9:22 AM, Andrew Christianson <andrew.christianson@
> nextcentury.com> wrote:
> >>
> >> Got it. So the crux of the problem is porting from v3 to v4, plus the
> added uncertainty of the C++ v4 target.
> >>
> >> I'm assuming that NiFi wants to eventually get onto v4 anyway. If
> that's the case, then porting to v4 is probably the ticket. Are there any
> concrete plans to do so in the NiFi mother project yet?
> >>
> >> -Andy
> >> ________________________________________
> >> From: Matt Burgess <mattyb149@gmail.com>
> >> Sent: Thursday, May 4, 2017 9:18:00 AM
> >> To: dev@nifi.apache.org
> >> Subject: Re: MiNiFi C++ Expression Language
> >>
> >> Correct, the current NiFi EL grammar is ANTLR3.
> >>
> >>
> >>> On May 4, 2017, at 9:12 AM, Andrew Christianson <andrew.christianson@
> nextcentury.com> wrote:
> >>>
> >>> Do I understand correctly that NiFi is currently using ANTLRv3?
> >>> ________________________________________
> >>> From: Matt Burgess <mattyb149@gmail.com>
> >>> Sent: Thursday, May 4, 2017 9:05:35 AM
> >>> To: dev@nifi.apache.org
> >>> Subject: Re: MiNiFi C++ Expression Language
> >>>
> >>> I haven't used Flex/Bison since a trivial example in college, so I'm
> not sure about the LOE for getting that set up, maybe there's a Maven-built
> project out there that we could look at for inspiration, but that seems
> unlikely :)
> >>>
> >>> An ANTLR4 refactor (assuming the C++ target is in good shape) would
> give us NiFi/MiNiFi EL compatibility (and full-featured EL support in
> MiNiFi C++), but we'd have to accept the risks of introducing bugs,
> regressions, etc. as a result of the refactor. Basically we'd just need to
> test the heck out of it on all platforms, which isn't a bad thing but adds
> to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for
> incremental development of a C/C++ based grammar.
> >>>
> >>>
> >>> On May 4, 2017, at 8:51 AM, Andrew Christianson <andrew.christianson@
> nextcentury.com> wrote:
> >>>
> >>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to
> refactor the existing grammar. Since the source and target for MiNiFi C++
> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
> are doing that with all the MiNiFi components already. The upside is being
> able to support EL incrementally as the grammar is developed. What do you
> think?
> >>>>
> >>>> This seems like a pragmatic approach. What's the level-of-effort
> required to do the initial grammar port and set up the build tooling? Less
> than refactoring for ANTLR4? I'm not as familiar with the EL grammar
> situation.
> >>>>
> >>>> -Andy
> >>>> ________________________________________
> >>>> From: Matt Burgess <mattyb149@gmail.com>
> >>>> Sent: Thursday, May 4, 2017 8:46:20 AM
> >>>> To: dev@nifi.apache.org
> >>>> Subject: Re: MiNiFi C++ Expression Language
> >>>>
> >>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to
> refactor the existing grammar. Since the source and target for MiNiFi C++
> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
> are doing that with all the MiNiFi components already. The upside is being
> able to support EL incrementally as the grammar is developed. What do you
> think?
> >>>>
> >>>> Regards,
> >>>> Matt
> >>>>
> >>>> [1] http://dinosaur.compilertools.net/
> >>>> [2] http://www.hwaci.com/sw/lemon/
> >>>> [3] http://www.colm.net/open-source/ragel/
> >>>>
> >>>>
> >>>>
> >>>> Sent from my iPhone
> >>>>> On May 4, 2017, at 8:13 AM, Marc P. <marc.parisi@gmail.com>
wrote:
> >>>>>
> >>>>> Andrew,
> >>>>> I am not aware of it being actively worked [1]. This would require
> using
> >>>>> ANTLR4, but I don't believe C++ support is well tested [2].  Someone
> can
> >>>>> correct me if I'm wrong, but there would have to be changes to both
> sides.
> >>>>> I attempted a quick straw man with grammars, but didn't take it
very
> far
> >>>>> after making initial changes to the grammar. It generated code,
but
> I'm
> >>>>> uncertain of cross platform compatibility with the expression
> language. If
> >>>>> that's not expected or required that will remove some limitations
as
> a
> >>>>> result of moving to ANTLR4.
> >>>>>
> >>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
> >>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-
> c-target-is-here
> >>>>>
> >>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
> >>>>> andrew.christianson@nextcentury.com> wrote:
> >>>>>
> >>>>>> All,
> >>>>>>
> >>>>>> I see that we do not have support for the expression language
yet in
> >>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is
there
> an ETA?
> >>>>>> If no one is working on it, is there a general plan for how
it
> should be
> >>>>>> implemented? I think I recall seeing references to ANTLR
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message