metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-1761) Allow a grok statement to be applied to each line in a file.
Date Wed, 03 Oct 2018 14:45:00 GMT

    [ https://issues.apache.org/jira/browse/METRON-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16637040#comment-16637040
] 

ASF GitHub Bot commented on METRON-1761:
----------------------------------------

Github user ottobackwards commented on the issue:

    https://github.com/apache/metron/pull/1184
  
    @mmiklavc I looked through the validation stuff more, I think that validation is the way
to go here.  The grok parser will add invalid message for each exception, parser failure,
and then in the validation call fail those messages.  It will have to be done so that the
returned message makes sense when it is sent to the error topic.  
    
    What do you think?   @cestella ?


> Allow a grok statement to be applied to each line in a file.
> ------------------------------------------------------------
>
>                 Key: METRON-1761
>                 URL: https://issues.apache.org/jira/browse/METRON-1761
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Laurens Vets
>            Assignee: Otto Fowler
>            Priority: Minor
>
> Make grok work where each line in incoming logs is a separate unit to be parsed.
> This would for instance allow NiFi to pick up log files (whereby each line is to be parsed
separately) and send them to Metron without having to split the content.
> Example content of a log file where a grok statement needs to be applied to each line:
> {code:java}
> 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000073
0.001048 0.000057 200 200 0 29 "GET http://www.example.com:80/ HTTP/1.1" "curl/7.38.0" - -
> 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000086
0.001048 0.001337 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1" "curl/7.38.0" DHE-RSA-AES128-SHA
TLSv1.2
> 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.001069
0.000028 0.000041 - - 82 305 "- - - " "-" - -
> 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.001065
0.000015 0.000023 - - 57 502 "- - - " "-" ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message