metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-1761) Allow a grok statement to be applied to each line in a file.
Date Thu, 04 Oct 2018 15:38:00 GMT

    [ https://issues.apache.org/jira/browse/METRON-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638388#comment-16638388
] 

ASF GitHub Bot commented on METRON-1761:
----------------------------------------

Github user mmiklavc commented on the issue:

    https://github.com/apache/metron/pull/1184
  
    @ottobackwards That sounds reasonable to me. One thing I would definitely add to the documentation
for this is some comment about how it changes processing semantics and might require additional
tuning. Since we now have batch sizing specified for the writers to Kafka, there may be a
little bit of tweaking there, but I'm just not sure.


> Allow a grok statement to be applied to each line in a file.
> ------------------------------------------------------------
>
>                 Key: METRON-1761
>                 URL: https://issues.apache.org/jira/browse/METRON-1761
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Laurens Vets
>            Assignee: Otto Fowler
>            Priority: Minor
>
> Make grok work where each line in incoming logs is a separate unit to be parsed.
> This would for instance allow NiFi to pick up log files (whereby each line is to be parsed
separately) and send them to Metron without having to split the content.
> Example content of a log file where a grok statement needs to be applied to each line:
> {code:java}
> 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000073
0.001048 0.000057 200 200 0 29 "GET http://www.example.com:80/ HTTP/1.1" "curl/7.38.0" - -
> 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000086
0.001048 0.001337 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1" "curl/7.38.0" DHE-RSA-AES128-SHA
TLSv1.2
> 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.001069
0.000028 0.000041 - - 82 305 "- - - " "-" - -
> 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.001065
0.000015 0.000023 - - 57 502 "- - - " "-" ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message