nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Petronic <markpetro...@gmail.com>
Subject Using InvokeHTTP to send GET but without content
Date Sat, 07 Nov 2015 21:57:22 GMT
I thought this seemed like a simple plan... I wanted to send an audit
message to a REST server every time I process every file in my flow. The
sub flow in question is:

+---------------+   +--------------+           +---------+
| UnpackContent +-->+ MergeContent +--merged+->+ PutFile |
+---------------+   +------+-------+           +---------+
                           |
                           v original
                   +-------+-------+    +----------+
                   |UpdateAttribute+--->+InvokeHTTP|
                   +---------------+    +----------+

I want to record every zip file unpacked so I feed (original) into
UpdateAttribute where I create a message like "Processed: ${ZipPath}",
where ${ZipPath} was added as an attribute earlier in the flow when I pull
in the zip files to process. I also want to send a another message from
PutFile once the file is delivered but my drawing does not show that part
yet. Same concept, hit a REST API to record the event. I also want to do
this sort of 'general' audit logging for other parts of the flow. For
example, I have a bunch of conditions/rule/actions to conditionally control
various processing. They are set up for files and content types of files
that I am aware of today. So, the [match] flow from some upstream
UpdateAttribute will flow the files that match the definitions. But, if
some unsuspecting file gets ingested, then it might NOT match a rule. So, I
want to route the [unmatched] into this audit logging where I can then see
this and decide what to do to handle this new file type. I know there is
providence but I want long term audits and don't need to retain the flow
file content and, frankly, it is just easier to work with plain audit logs
(grep/awk/etc), IMO, and for my needs at this time.

Problem is, InvokeHTTP never invokes. Why not? I think it would be nice if
InvokeHTTP would hit the URL without a flow file, as in my case, I never
want to actually send any body content but rather access attributes to
build up a URL with param=value pairs to send. I am probably off base here.
Any ideas? I saw this exchange but this seems kludgy using the generate
flow file just to "trick" Invoke HTTP into doing something.

https://mail-archives.apache.org/mod_mbox/nifi-users/201509.mbox/%3CCALJK9a4U29Ua9zVT9oPCYHMg=k4Vneao30EMszRe3yPkz6xxJQ@mail.gmail.com%3E

I am trying to adapt to the "Nifi" paradigm but feel like I am swimming
upstream at times.

Thanks

Mime
View raw message