Currently I am using flume for data ingestion and my use case as follows
Log repository--------log copy Script-----à Staging directory for copied logs
Staging directory for copied logs folder structure----Machine1log----a.log
Flume will copy these logs and replicate same structure in HDFS cluster. Beginning with which is :
And creates 0 byte dummy file with same name so that Script wont copy the same log again as it find 0 byte file already existing in source directory.
Can we do same things with apache nifi?
Keeping in mind two goals- same folder structure in HDFS and after moving file to HDFS it should crete 0 byte dummy file in source directory.
With Best Regards,
Philips Innovation campus