metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-381) Add support for multiple reducers in pcap_query.sh
Date Wed, 07 Sep 2016 21:51:20 GMT

    [ https://issues.apache.org/jira/browse/METRON-381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15471898#comment-15471898
] 

ASF GitHub Bot commented on METRON-381:
---------------------------------------

Github user dlyle65535 commented on the issue:

    https://github.com/apache/incubator-metron/pull/217
  
    +1, looks good!


> Add support for multiple reducers in pcap_query.sh
> --------------------------------------------------
>
>                 Key: METRON-381
>                 URL: https://issues.apache.org/jira/browse/METRON-381
>             Project: Metron
>          Issue Type: New Feature
>            Reporter: Casey Stella
>            Assignee: Casey Stella
>
> As a stopgap, the pcap_query.sh MapReduce job uses a single reducer to ensure that the
packets captured come in the correct order.  This has issues on large queries because it forces
all of the data to go through one reducer.  Instead, we should have a custom partitioner that
will allow us to use multiple reducers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message