metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Foley <ma...@apache.org>
Subject Re: [DISCUSS] New Stellar Functions
Date Mon, 10 Apr 2017 19:07:44 GMT
Hi Kyle,
For now at least, it seems to me that something with as much core usefulness, and presumably
non-customer-specific, as string concat should go into metron-common.  That said, I suspect
Mike is right that JOIN would be sufficient.  But if that fits your use case poorly, just
say so.

--Matt

On 4/10/17, 10:13 AM, "Michael Miklavcic" <michael.miklavcic@gmail.com> wrote:

    Hey Kyle,
    
    It probably belongs here -
    https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
    There is an existing JOIN function for strings - might this suit your
    needs? I didn't see a unit test for it, so it would probably be good for us
    to backfill with a test here as well. I can submit a PR for it, or if
    you're already in that code, you're welcome to also -
    https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-common/src/test/java/org/apache/metron/common/dsl/functions/StringFunctionsTest.java
    
    e.g.
    Object joined = run("JOIN(['A','B','C','D'], ':')",new HashedMap());
    System.out.println(joined);
    Object joined2 = run("JOIN(['A','B','C','D'], '')",new HashedMap());
    System.out.println(joined2);
    
    Output I get is:
    A:B:C:D
    ABCD
    
    AFA where to put these functions, I believe we have a number of options now
    that we have the ability to add to the storm topology classpath and
    sideload jars.
    - https://github.com/apache/incubator-metron/pull/204
    - https://github.com/apache/incubator-metron/pull/468
    
    I think that if the functions are unique to a customer, they should
    probably be built as a stand-alone Maven project. I believe Otto is working
    on this ATM if I'm not mistaken. If there is universal (across all
    functions in the system, whether parsing, analytics or otherwise) then they
    should probably go in with the dsl package in metron-common. At some point
    we might want to make Stellar its own module, but there is some work there.
    
    Best,
    Mike
    
    
    On Sun, Apr 9, 2017 at 2:26 PM, Kyle Richardson <kylerichardson2@gmail.com>
    wrote:
    
    > I have the need for a new Stellar function to perform string concatenation.
    > I have it implemented but am curious about where new functions should live
    > given the new capabilities around 3rd party Stellar function libraries.
    >
    > So, I guess my question is, should this function live in:
    > 1) metron-common with the other string functions
    > 2) another metron project
    > 3) as a standalone project and not part of the metron source tree
    >
    > While I'm specifically asking about this case, I think it's also worthwhile
    > that we think about where other new functions should live in the long term.
    >
    > Thanks!
    >
    > -Kyle
    >
    



Mime
View raw message