metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-189) Add the ability to do global validations on messages passing through the parser.
Date Wed, 01 Jun 2016 23:34:59 GMT

    [ https://issues.apache.org/jira/browse/METRON-189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311360#comment-15311360
] 

ASF GitHub Bot commented on METRON-189:
---------------------------------------

Github user merrimanr commented on a diff in the pull request:

    https://github.com/apache/incubator-metron/pull/138#discussion_r65461745
  
    --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java
---
    @@ -140,18 +142,25 @@ public void execute(Tuple tuple) {
         try {
           boolean ackTuple = true;
           if(sensorParserConfig != null) {
    +        List<FieldValidator> fieldValidations = getConfigurations().getFieldValidations();
             List<JSONObject> messages = parser.parse(originalMessage);
             for (JSONObject message : messages) {
               if (parser.validate(message)) {
    -            if (filter != null && filter.emitTuple(message)) {
    -              ackTuple = !isBulk;
    -              message.put(Constants.SENSOR_TYPE, getSensorType());
    -              for (FieldTransformer handler : sensorParserConfig.getFieldTransformations())
{
    -                if (handler != null) {
    -                  handler.transformAndUpdate(message, sensorParserConfig.getParserConfig());
    +            if(!isGloballyValid(message, fieldValidations)) {
    +              message.put(Constants.SENSOR_TYPE, getSensorType()+ ".invalid");
    +              collector.emit(Constants.INVALID_STREAM, new Values(message));
    +            }
    +            else if (filter != null && filter.emitTuple(message)) {
    +              if (filter != null && filter.emitTuple(message)) {
    --- End diff --
    
    Isn't this a duplicate of the previous line?


> Add the ability to do global validations on messages passing through the parser.
> --------------------------------------------------------------------------------
>
>                 Key: METRON-189
>                 URL: https://issues.apache.org/jira/browse/METRON-189
>             Project: Metron
>          Issue Type: New Feature
>            Reporter: Casey Stella
>            Assignee: Casey Stella
>
> Allow the user to specify field level or message level validations to ensure messages
coming from the parser are valid.  For instance, allow the ability ensure that a field is
an IPv4 address.
> If a field is invalid, send to a separate stream from the parser bolt.  Follow-on work
should be done to send this stream to the index for after-the-fact inspection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message