nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bronislav Jitnikov <shadi...@gmail.com>
Subject Fetching data from database on cluster.
Date Tue, 27 Nov 2018 16:03:57 GMT
I have small cluster with 3 instances of NiFi, and found some problem, as I
think.
Processor QueryDatabaseTable set to work on PrimaryNode only and Concurent
tasks set to 1. Run Schedule set to large value (something like 20
minutes), so I expect only one execution at a time. While query is
executed, primary node changed and new Task started on new primary node. So
I see two ways to resolve this problem:
1. Create some sort of lock on QueryDatabaseTable (maybe custom proc that
lock run across the cluster StateManager)
2. Add some check in  connectableTask.invoke() (Better for me because I
have similar problems with get data from REST).

May be I miss something: So any help and ideas would be appreciated.

Bronislav Zhitnikov

PS: and sorry for my bad English.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message