nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <mattyb...@gmail.com>
Subject Re: MiNiFi C++ Expression Language
Date Thu, 04 May 2017 15:52:05 GMT
Good point, we should select a parser generator that produces tight code. Having said that,
the MVP grammar is so simple that it could easily be put into a parser generator DSL later,
and the MVP implementation could be hand-written for the first iteration (I'm picturing a
couple dozen lines of code, but that's just a knee-jerk estimation).


> On May 4, 2017, at 11:25 AM, Marc <phrocker@apache.org> wrote:
> 
> Andy,
>    Depending on the type of parser chosen you may run into memory
> limitations on devices that run MiNiFi-C++. Since that's a deployment
> concern -- and one that few will face, I would add that we can make this a
> conditional build and only including the necessary components within the
> binary if a build option is not specified to exclude these components.
> 
> On Thu, May 4, 2017 at 11:14 AM, Andrew Christianson <
> andrew.christianson@nextcentury.com> wrote:
> 
>> The island grammar sounds very appealing for an MVP. Simple to implement,
>> yet it covers a very common EL use case (dynamically inserting attr vals
>> into property values). If we have general consensus I would love to see a
>> MINIFI JIRA ticket added for this.
>> 
>> -Andy
>> ________________________________________
>> From: Matt Burgess <mattyb149@gmail.com>
>> Sent: Thursday, May 4, 2017 11:09:46 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>> 
>> What you have sounds good to me. IMO minimum viable product would be an
>> island grammar (meaning you can have any characters outside a ${}
>> expression) and inside would support an attribute name. Next steps could be
>> nested expressions and/or support for functions, added piecemeal as the
>> contributor sees fit. I wasn't involved in the early development of the EL
>> grammar, so maybe someone who was has some thoughts on a natural evolution.
>> 
>> Regards,
>> Matt
>> 
>>> On May 4, 2017, at 11:00 AM, Andrew Christianson <andrew.christianson@
>> nextcentury.com> wrote:
>>> 
>>> My bad, what does the sketch of the plan *look like*?
>>> 
>>> -Andy
>>> ________________________________________
>>> From: Andrew Christianson
>>> Sent: Thursday, May 4, 2017 10:59:07 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>> 
>>> What does the sketch of the plan to do the separate implementation?
>> Write a flex/bison grammar, hook it into the cmake build, and start using
>> it? Any constraints on features or syntax that this separate implementation
>> must support?
>>> 
>>> -Andy
>>> ________________________________________
>>> From: Matt Burgess <mattyb149@gmail.com>
>>> Sent: Thursday, May 4, 2017 9:37:34 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>> 
>>> No plans that I know of. In the meantime, EL support for MiNiFi is kind
>> of held hostage, so maybe the separate implementation is more viable in the
>> nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever
>> exists by then with the cross-platform ANTLR target generation, and test
>> the whole kit and caboodle.
>>> 
>>> Anyone out there familiar with the aforementioned tools (or willing to
>> hand-roll one)? What do you think about this "EL bootstrapping" approach?
>>> 
>>> Thanks,
>>> Matt
>>> 
>>>> On May 4, 2017, at 9:22 AM, Andrew Christianson <andrew.christianson@
>> nextcentury.com> wrote:
>>>> 
>>>> Got it. So the crux of the problem is porting from v3 to v4, plus the
>> added uncertainty of the C++ v4 target.
>>>> 
>>>> I'm assuming that NiFi wants to eventually get onto v4 anyway. If
>> that's the case, then porting to v4 is probably the ticket. Are there any
>> concrete plans to do so in the NiFi mother project yet?
>>>> 
>>>> -Andy
>>>> ________________________________________
>>>> From: Matt Burgess <mattyb149@gmail.com>
>>>> Sent: Thursday, May 4, 2017 9:18:00 AM
>>>> To: dev@nifi.apache.org
>>>> Subject: Re: MiNiFi C++ Expression Language
>>>> 
>>>> Correct, the current NiFi EL grammar is ANTLR3.
>>>> 
>>>> 
>>>>> On May 4, 2017, at 9:12 AM, Andrew Christianson <andrew.christianson@
>> nextcentury.com> wrote:
>>>>> 
>>>>> Do I understand correctly that NiFi is currently using ANTLRv3?
>>>>> ________________________________________
>>>>> From: Matt Burgess <mattyb149@gmail.com>
>>>>> Sent: Thursday, May 4, 2017 9:05:35 AM
>>>>> To: dev@nifi.apache.org
>>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>> 
>>>>> I haven't used Flex/Bison since a trivial example in college, so I'm
>> not sure about the LOE for getting that set up, maybe there's a Maven-built
>> project out there that we could look at for inspiration, but that seems
>> unlikely :)
>>>>> 
>>>>> An ANTLR4 refactor (assuming the C++ target is in good shape) would
>> give us NiFi/MiNiFi EL compatibility (and full-featured EL support in
>> MiNiFi C++), but we'd have to accept the risks of introducing bugs,
>> regressions, etc. as a result of the refactor. Basically we'd just need to
>> test the heck out of it on all platforms, which isn't a bad thing but adds
>> to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for
>> incremental development of a C/C++ based grammar.
>>>>> 
>>>>> 
>>>>> On May 4, 2017, at 8:51 AM, Andrew Christianson <andrew.christianson@
>> nextcentury.com> wrote:
>>>>> 
>>>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job
to
>> refactor the existing grammar. Since the source and target for MiNiFi C++
>> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
>> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
>> are doing that with all the MiNiFi components already. The upside is being
>> able to support EL incrementally as the grammar is developed. What do you
>> think?
>>>>>> 
>>>>>> This seems like a pragmatic approach. What's the level-of-effort
>> required to do the initial grammar port and set up the build tooling? Less
>> than refactoring for ANTLR4? I'm not as familiar with the EL grammar
>> situation.
>>>>>> 
>>>>>> -Andy
>>>>>> ________________________________________
>>>>>> From: Matt Burgess <mattyb149@gmail.com>
>>>>>> Sent: Thursday, May 4, 2017 8:46:20 AM
>>>>>> To: dev@nifi.apache.org
>>>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>>> 
>>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to
>> refactor the existing grammar. Since the source and target for MiNiFi C++
>> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
>> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
>> are doing that with all the MiNiFi components already. The upside is being
>> able to support EL incrementally as the grammar is developed. What do you
>> think?
>>>>>> 
>>>>>> Regards,
>>>>>> Matt
>>>>>> 
>>>>>> [1] http://dinosaur.compilertools.net/
>>>>>> [2] http://www.hwaci.com/sw/lemon/
>>>>>> [3] http://www.colm.net/open-source/ragel/
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Sent from my iPhone
>>>>>>> On May 4, 2017, at 8:13 AM, Marc P. <marc.parisi@gmail.com>
wrote:
>>>>>>> 
>>>>>>> Andrew,
>>>>>>> I am not aware of it being actively worked [1]. This would require
>> using
>>>>>>> ANTLR4, but I don't believe C++ support is well tested [2]. 
Someone
>> can
>>>>>>> correct me if I'm wrong, but there would have to be changes to
both
>> sides.
>>>>>>> I attempted a quick straw man with grammars, but didn't take
it very
>> far
>>>>>>> after making initial changes to the grammar. It generated code,
but
>> I'm
>>>>>>> uncertain of cross platform compatibility with the expression
>> language. If
>>>>>>> that's not expected or required that will remove some limitations
as
>> a
>>>>>>> result of moving to ANTLR4.
>>>>>>> 
>>>>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>>>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-
>> c-target-is-here
>>>>>>> 
>>>>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>>>>>> andrew.christianson@nextcentury.com> wrote:
>>>>>>> 
>>>>>>>> All,
>>>>>>>> 
>>>>>>>> I see that we do not have support for the expression language
yet in
>>>>>>>> MiNiFi C++. Is anyone actively working on this, and if so,
is there
>> an ETA?
>>>>>>>> If no one is working on it, is there a general plan for how
it
>> should be
>>>>>>>> implemented? I think I recall seeing references to ANTLR
>> 

Mime
View raw message