nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Grande <>
Subject Re: Nifi processor last execution time
Date Fri, 13 Nov 2015 19:56:56 GMT
There are 2 approached NiFi processors follow today.

E.g. a new TailFile saves it in a local state file (relative to NiFi root, in the ./conf/

Others leverage a DistributedCache service (it's actually more like a RemoteCache, not a distributed/replicated),
which they declare as a requirement in the properties. This also has a benefit of being able
to share the state between nodes and, optionally, persist the cache to the disk to be able
to re-hydrate it on restart.


On 11/13/15, 10:09 AM, "Sean Busbey" <> wrote:

>For now I would store this information in HDFS. Alternatively, if you have
>it ZooKeeper would also work.
>On Thu, Nov 12, 2015 at 10:45 PM, Sam Kamau <> wrote:
>> I am working on a processor that loads data to HDFS for an RDBMS using
>> ExecuteSQL processor. The data has a last update time that I would like to
>> use to load records that have changed since the last load.
>> How best can I go about saving the last execution time so I can use it as a
>> filter in the query.
>> Thanks
View raw message