nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Burgess <mattyb...@gmail.com>
Subject Re: Log4j/logback parser via syslog
Date Fri, 12 Feb 2016 16:08:00 GMT
That is pretty much the list although native Python and Ruby are not supported, only Jython
and JRuby. The link in Bryan’s email is to a single post but the blog itself has about 5
so far: http://funnifi.blogspot.com.

The two scripting processors are ExecuteScript and InvokeScriptedProcessor, the former is
lighter-weight and lets you interact with incoming flow files, create new ones, add attributes,
etc. The latter lets you script a full Processor implementation for when you need more control
over the processor lifecycle, relationships, properties, etc.

A quick Google search didn’t turn up a good non-GUI open-source log4J parser, but if there
is one out there (for the JVM), you can download it and use the Module Directory property
to make it available to your script. There’s a post on my blog about that too: http://funnifi.blogspot.com/2016/02/executescript-using-modules.html

Regards,
Matt

From:  Bryan Bende <bbende@gmail.com>
Reply-To:  <users@nifi.apache.org>
Date:  Friday, February 12, 2016 at 10:55 AM
To:  <users@nifi.apache.org>
Subject:  Re: Log4j/logback parser via syslog

I believe groovy, python, jython, jruby, ruby, javascript, and lua.

The associated JIRA is here:
https://issues.apache.org/jira/browse/NIFI-210

There are some cool blogs about them here:
http://funnifi.blogspot.com/2016/02/executescript-processor-hello-world.html

-Bryan

On Fri, Feb 12, 2016 at 10:48 AM, Madhukar Thota <madhukar.thota@gmail.com> wrote:
Thanks Bryan. I will look into ExtractText processor.

Do you know what scripting languages are supported with new processors?

-Madhu

On Fri, Feb 12, 2016 at 9:27 AM, Bryan Bende <bbende@gmail.com> wrote:
Hello,

Currently there are no built in processors to parse log formats, but have you taken a look
at the ExtractText processor [1]? 

If you can come up with a regular expression for whatever you are trying to extract, then
you should be able to use ExtractText.

Other options... 

You could write a custom processor, but this sounds like it might be overkill for your scenario.
In the next release (hopefully out in a few days) there will be two new processors that support
scripting languages. It may be easier to use a scripting language to manipulate/parse the
text. 

Thanks,

Bryan

[1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ExtractText/index.html


On Fri, Feb 12, 2016 at 12:16 AM, Madhukar Thota <madhukar.thota@gmail.com> wrote:
Hi 

I am very new to Apache Nifi and just started learning about how to use it.

We have a requirement where we need to parse log4j/logback pattern messages coming from SyslogAppenders
via Syslog udp. I can read the standard syslog messages, but how can i further extract log4j/logback
messages  from syslog body.

Is there any log parsers( log4j/logback/Apache access log format) available in apache nifi?


Any help on this much appreciated. 

Thanks in Advance.






Mime
View raw message