nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Frank, Daniel" <Daniel.Fr...@bauermedia.com>
Subject Nifi as a gateway for our transfer system
Date Tue, 13 Jun 2017 17:17:36 GMT
Hello,

I just found Apache Nifi recently and I decided to use it to take care of our ftp transfer
system, which basically means to monitor the incoming data and generate Flowfiles for each
incoming file.
The basic flow I have in mind is:
1. GetFile to monitor the incoming directory
2. Lookup the actions for each Flowfile based on their path and filename attributes in a database
and store them as an attribute
3. Figure out the next action to do by looking at that attribute and route the Flowfile into
the correct flow implementing that action (e.g. execute a command or send an email)
4. After that action has completed, send the Flowfile back to step 3 if there are more actions
that need to be done

I'm somewhat stumped on how to implement that in a sensible way with Nifi. Step 2 will result
in something like the following json and I guess I have to write my own processor for that:
[ { "unzip": null }, {"email": {"to": "somebody@example.com", "cc": "someone@example.com",
"withAttachment": false}, { "execute": "/bin/true" }, { "execute": "/bin/anothercommand "
} ]

Now I can store that as an attribute, but apparently EvaluateJsonPath like all other Evaluate*
processors are unable to parse attributes and storing them as content is not possible, as
the existing content is what I have to work on later.
I could also store just the next action type (unzip, email, execute) as an Attribute for the
RouteOnAttribute processor, but that means I need to run the Flowfile through yet another
script and even more scripts once it's been routed to the flow implementing the action.

Due to the amount of scripts involved I'm beginning to question if Nifi is the right tool
for this job. Am I missing an obvious solution to this?

Regards,
Daniel Frank


Mime
View raw message