metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-1681) Decouple the ParserBolt from the Parse execution logic
Date Wed, 03 Oct 2018 20:12:00 GMT

    [ https://issues.apache.org/jira/browse/METRON-1681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16637474#comment-16637474
] 

ASF GitHub Bot commented on METRON-1681:
----------------------------------------

Github user merrimanr commented on the issue:

    https://github.com/apache/metron/pull/1213
  
    Thanks for pointing me to that @mmiklavc, I had not looked at it yet.  I am in agreement
that we should do our best to match other classes and patterns we already have.  I will continue
studying that to see where we can make things more consistent.  If you can provide some examples
that demonstrate your ideas that would help me understand it better.  
    
    From what I can tell, the strategy pattern was used in the UnifiedEnrichmentBolt because
there are 2 separate strategies for enriching a message:  enrichment and threat intel.  Do
we need the strategy pattern here since there is only one parser running strategy?  We could
implement this as a strategy pattern in anticipation of one day needing another parser running
strategy.  Do you think it's worth doing now or later when we need it?
    
    The other difference I see is that Future objects are used to join the different messages
after they are processed in parallel.  I believe this is done because all enrichments/threat
intel must be done before the message can be pieced back together and sent along.  In this
case we don't have that limitation.  The documents that are parsed do not depend on each other
and can be passed along as soon as they are processed.  We are set up for post-processing
these documents in parallel but I see that as a low-latency operation and I'm not sure it's
worth the extra overhead.
    



> Decouple the ParserBolt from the Parse execution logic
> ------------------------------------------------------
>
>                 Key: METRON-1681
>                 URL: https://issues.apache.org/jira/browse/METRON-1681
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Justin Leet
>            Priority: Major
>
> Per discussion on https://github.com/apache/metron/pull/1099, there are concerns about
the ParserBolt needed some refactoring.  The discussion didn't hold the PR up, but it was
generally agreed that we should decouple some of the initialization and execution logic.
> This also aids us in integrating with other systems such as NiFi or Spark.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message