nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tzhu <js.tianlu...@gmail.com>
Subject NiFi - replace custom script by available processors
Date Tue, 05 Dec 2017 21:12:54 GMT
Hi,

My task is to analyze some log files to count the number of certain string
occurrences and record the number in a SQL table. If the table does not
exist, NiFi should create a new SQL table and put the data in. I have the
process set up as the following picture:
<http://apache-nifi-users-list.2361937.n4.nabble.com/file/t339/nifi.jpg> 

In ExecuteScript, I write a few lines using callback to read the content and
write a new flowfile based on the result. The output is a string which can
be used as a SQL query. If PutSQL is successful, then the process
terminates; if it failed, then it goes to another custom script which
contains the "create table" query combined with the information from
previous flowfile (the insert query before).

I want to know if I can improve the process by replacing the ExecuteScript
processors by other processors.
I have two failed approaches:
1. In the analyze part, use split text to split the contents into lines,
then use RouteOnContent to find the lines with the desired string, then use
updateCounter to record the number. But then I stuck with how to get the
counter number into flowfile.
2. To put flowfile into SQL table, I want to use other format such as JSON,
then use convertJSONToSQL to create the SQL query. But I got error message
saying "1st arg can't be coerced to int". Maybe I can't use dictionary as
JSON file.

Or maybe there are some better ways improve the process. Any help is
appreciated!

Thanks,
Tina



--
Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

Mime
View raw message