nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan <>
Subject Use Case Validation
Date Wed, 23 Dec 2015 15:52:41 GMT
I've recently found NiFi and have been playing around with it locally for a day or so to assess
whether it would be a good fit for the following use case:
1. I'm tasked with gathering log files from 100s of machines from a predetermined directory
structure local to the machine (e.g. /log/appname/ or c:\log\appname) which may be Linux or
Windows2. File names include date (e.g. appname_20151223.log)3. The log file is structured
as JSON - each line of the file is a JSON object4. The JSON object in each file includes data
that determines where to route the message5. Each message should be routed to one of several
Azure Event Hubs based on #4
Would I set up a single NiFi cluster to do this, or would I set up what would essentially
be 100 NiFi clusters if I have 100 machines from which I want to gather logs from their local
/log/appname directory?
Thanks - this looks like a very well thought out project!
View raw message