nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <mattyb...@gmail.com>
Subject Re: MiNiFi C++ Expression Language
Date Thu, 04 May 2017 15:09:46 GMT
What you have sounds good to me. IMO minimum viable product would be an island grammar (meaning
you can have any characters outside a ${} expression) and inside would support an attribute
name. Next steps could be nested expressions and/or support for functions, added piecemeal
as the contributor sees fit. I wasn't involved in the early development of the EL grammar,
so maybe someone who was has some thoughts on a natural evolution.

Regards,
Matt

> On May 4, 2017, at 11:00 AM, Andrew Christianson <andrew.christianson@nextcentury.com>
wrote:
> 
> My bad, what does the sketch of the plan *look like*?
> 
> -Andy
> ________________________________________
> From: Andrew Christianson
> Sent: Thursday, May 4, 2017 10:59:07 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
> 
> What does the sketch of the plan to do the separate implementation? Write a flex/bison
grammar, hook it into the cmake build, and start using it? Any constraints on features or
syntax that this separate implementation must support?
> 
> -Andy
> ________________________________________
> From: Matt Burgess <mattyb149@gmail.com>
> Sent: Thursday, May 4, 2017 9:37:34 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
> 
> No plans that I know of. In the meantime, EL support for MiNiFi is kind of held hostage,
so maybe the separate implementation is more viable in the nearer term. If/When the ANTLR4
upgrade happens, we could replace whatever exists by then with the cross-platform ANTLR target
generation, and test the whole kit and caboodle.
> 
> Anyone out there familiar with the aforementioned tools (or willing to hand-roll one)?
What do you think about this "EL bootstrapping" approach?
> 
> Thanks,
> Matt
> 
>> On May 4, 2017, at 9:22 AM, Andrew Christianson <andrew.christianson@nextcentury.com>
wrote:
>> 
>> Got it. So the crux of the problem is porting from v3 to v4, plus the added uncertainty
of the C++ v4 target.
>> 
>> I'm assuming that NiFi wants to eventually get onto v4 anyway. If that's the case,
then porting to v4 is probably the ticket. Are there any concrete plans to do so in the NiFi
mother project yet?
>> 
>> -Andy
>> ________________________________________
>> From: Matt Burgess <mattyb149@gmail.com>
>> Sent: Thursday, May 4, 2017 9:18:00 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>> 
>> Correct, the current NiFi EL grammar is ANTLR3.
>> 
>> 
>>> On May 4, 2017, at 9:12 AM, Andrew Christianson <andrew.christianson@nextcentury.com>
wrote:
>>> 
>>> Do I understand correctly that NiFi is currently using ANTLRv3?
>>> ________________________________________
>>> From: Matt Burgess <mattyb149@gmail.com>
>>> Sent: Thursday, May 4, 2017 9:05:35 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>> 
>>> I haven't used Flex/Bison since a trivial example in college, so I'm not sure
about the LOE for getting that set up, maybe there's a Maven-built project out there that
we could look at for inspiration, but that seems unlikely :)
>>> 
>>> An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi
EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the
risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just
need to test the heck out of it on all platforms, which isn't a bad thing but adds to the
LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development
of a C/C++ based grammar.
>>> 
>>> 
>>> On May 4, 2017, at 8:51 AM, Andrew Christianson <andrew.christianson@nextcentury.com>
wrote:
>>> 
>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor
the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative
could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining
two grammars, but we are doing that with all the MiNiFi components already. The upside is
being able to support EL incrementally as the grammar is developed. What do you think?
>>>> 
>>>> This seems like a pragmatic approach. What's the level-of-effort required
to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4?
I'm not as familiar with the EL grammar situation.
>>>> 
>>>> -Andy
>>>> ________________________________________
>>>> From: Matt Burgess <mattyb149@gmail.com>
>>>> Sent: Thursday, May 4, 2017 8:46:20 AM
>>>> To: dev@nifi.apache.org
>>>> Subject: Re: MiNiFi C++ Expression Language
>>>> 
>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor
the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative
could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining
two grammars, but we are doing that with all the MiNiFi components already. The upside is
being able to support EL incrementally as the grammar is developed. What do you think?
>>>> 
>>>> Regards,
>>>> Matt
>>>> 
>>>> [1] http://dinosaur.compilertools.net/
>>>> [2] http://www.hwaci.com/sw/lemon/
>>>> [3] http://www.colm.net/open-source/ragel/
>>>> 
>>>> 
>>>> 
>>>> Sent from my iPhone
>>>>> On May 4, 2017, at 8:13 AM, Marc P. <marc.parisi@gmail.com> wrote:
>>>>> 
>>>>> Andrew,
>>>>> I am not aware of it being actively worked [1]. This would require using
>>>>> ANTLR4, but I don't believe C++ support is well tested [2].  Someone
can
>>>>> correct me if I'm wrong, but there would have to be changes to both sides.
>>>>> I attempted a quick straw man with grammars, but didn't take it very
far
>>>>> after making initial changes to the grammar. It generated code, but I'm
>>>>> uncertain of cross platform compatibility with the expression language.
If
>>>>> that's not expected or required that will remove some limitations as
a
>>>>> result of moving to ANTLR4.
>>>>> 
>>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>>>> 
>>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>>>> andrew.christianson@nextcentury.com> wrote:
>>>>> 
>>>>>> All,
>>>>>> 
>>>>>> I see that we do not have support for the expression language yet
in
>>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there
an ETA?
>>>>>> If no one is working on it, is there a general plan for how it should
be
>>>>>> implemented? I think I recall seeing references to ANTLR

Mime
View raw message