metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Justin Leet (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (METRON-1644) Support parser chaining
Date Wed, 05 Sep 2018 15:22:00 GMT

     [ https://issues.apache.org/jira/browse/METRON-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Justin Leet updated METRON-1644:
--------------------------------
    Fix Version/s: 0.6.0

> Support parser chaining
> -----------------------
>
>                 Key: METRON-1644
>                 URL: https://issues.apache.org/jira/browse/METRON-1644
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Casey Stella
>            Assignee: Casey Stella
>            Priority: Major
>             Fix For: 0.6.0
>
>
> Currently we have only one layer of parsing prior to enrichment, but often real data
is more complex.  For instance, often data is wrapped in an envelope (e.g. syslog data which
contains a field which needs to be parsed).
> This Jira allows us to support a DAG of parsers prior to enrichment by allowing us to
provide a strategy for interpreting what is data and what is metadata in the parser bolt. 
This enables upstream parsers to pass in a JSON Blob which contains the metadata and have
the parser bolt choose which field is the data to be parsed, the remaining fields would be
considered metadata (and merged into the resulting data or not depending on our existing rules
for handling metadata).
>  
> To illustrate this better, I've provided a use-case with an example.  Note, this PR
depends on METRON-1643 and METRON-1642, so those should be reviewed prior to this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message