nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andre <andre-li...@fucs.org>
Subject Re: ParseCEF processor fails when Msg field exceed 1023 characters
Date Sat, 05 Jan 2019 00:57:22 GMT
Otto,

I considered that but never implemented it in NiFi as I had concerns people
would have the incorrect assumption the parser can help on those cases.

>From the top of my head, I have seen type mismatch, length overflows,
malformed CEF headers (the pipe delimited section of the CEF message). I
wouldn't be surprised there are a lot of other poorly formed messages.

Cheers

On Fri., 4 Jan. 2019, 23:40 Otto Fowler <ottobackwards@gmail.com wrote:

> Yeah, I went through the code after my mail last night.   Would having
> validation optional not be a valid setting?  How often would the output
> feeding this flow be invalid for some other reason?
>
>
>
> On January 4, 2019 at 04:33:54, Andre (andre-lists@fucs.org) wrote:
>
> Otto,
>
> The parser should fail in that case (and many others).
>
>
> https://github.com/fluenda/ParCEFone/blob/master/src/main/java/com/fluenda/parcefone/event/CefRev23.java#L327
>
> Cheers
>
> On Fri, Jan 4, 2019 at 10:05 AM Otto Fowler <ottobackwards@gmail.com>
> wrote:
>
>> Can I ask how you are sure it is the message size that is causing the
>> error?  The parser returns null for any error parsing, so the processor
>> doesn’t know what happened.  It could be that the message didn’t validate,
>> or something else.
>>
>> If the issues _is_ with the validator, then we could allow a property to
>> optionally call the parser with the do validate flag to false.
>>
>> Maybe you can create a jira with a sanitized example line that causes the
>> error?
>>
>>
>>
>>
>> On January 3, 2019 at 15:13:14, Felix McPherson (ljungpipare@yahoo.se)
>> wrote:
>>
>> Hi,
>> I'm using the ParseCEF processor to parse CEF message to Json format.
>> Unfortunately the ParseCEF processor fails for message/events that holds a
>> string in the Msg field that has more than 1023 character. According to the
>> CEF standard the Msg field in an event shall not exceed 1023 character. The
>> PARSECEF fails with:
>>
>> "Error
>> ParseCEF[id=...] Failed to parse...
>> ...as a CEF message; it does not conform to the CEF standard; routing to
>> failure.
>>
>> Any ideas on a workaround for this problem? I would prefer not having to
>> remove character in the Msg field string.
>> Regards,lj
>>
>>

Mime
View raw message