metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-1681) Decouple the ParserBolt from the Parse execution logic
Date Tue, 02 Oct 2018 20:11:00 GMT

    [ https://issues.apache.org/jira/browse/METRON-1681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636056#comment-16636056
] 

ASF GitHub Bot commented on METRON-1681:
----------------------------------------

Github user merrimanr commented on the issue:

    https://github.com/apache/metron/pull/1213
  
    The MessageParser.parseOptional call returns a list of JSONObjects which is then processed
(field transformations applied, messages validated, etc).  The difference is WHEN each processed
JSONObject is written to Kafka.  In the current ParserBolt each one is sent as soon as it
is processed because everything runs in ParserBolt.execute.  Since we are extracting that
logic outside of the ParserBolt, we either need to write them all after processing is done
by the abstraction or expose callbacks that the abstraction calls as it finishes processing
each one.
    
    I don't think it matters much either way.  I chose to use callbacks because it more closely
matches the flow of the current ParserBolt.  I will assume everyone is ok with callbacks and
will continue work on the suggestions made in the comments.


> Decouple the ParserBolt from the Parse execution logic
> ------------------------------------------------------
>
>                 Key: METRON-1681
>                 URL: https://issues.apache.org/jira/browse/METRON-1681
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Justin Leet
>            Priority: Major
>
> Per discussion on https://github.com/apache/metron/pull/1099, there are concerns about
the ParserBolt needed some refactoring.  The discussion didn't hold the PR up, but it was
generally agreed that we should decouple some of the initialization and execution logic.
> This also aids us in integrating with other systems such as NiFi or Spark.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message