nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuri Nikonovich <utagai...@gmail.com>
Subject How to configure site-to-site communication between nodes in one cluster.
Date Wed, 01 Jun 2016 10:06:44 GMT
Hi
I have the following flow:
Receive HTTP request -> Fetch data from db -> split it in chunks of fixed
size -> process each chunk and save it to Cassandra.

I've built a flow and it works perfectly on non-clustered setup. But when I
configured clustered setup
I found out that all heavy work is done only on one node. So if the flow
has started on node1 it will run to the end on node1. What I want to
achieve is to spread data chunks fetched from DB across the cluster in
order to process them in parallel, but it looks like Nifi doesn't send flow
files between nodes in a cluster.
As far as I understand, in order to make node send data to another node I
should create a remote process group and send data to this RPG. All
examples I could find on Internet describe RPGs as cluster-to-cluster
communication or remote node-to-cluster communication. So for my case, I
assume, have to create RPG pointing to the same cluster. Could you please
provide me a guide how to do this.


-- 
Regards,
Nikanovich Yury

Mime
View raw message