I was trying to write a simple case of using kafka connector. My setup involves using three nodes N1,N2 and N3.
N1 is the source and N2, N3 are the sink nodes in my case.
I am writing data to a text file(say input.txt) on Node N1 and using the standalone kafka connector I wish to see a text file with content similar to input.txt on the nodes N2 and N3.
I am using the REST API to make changes in topic name, file name and tasks.max.
However, during the experiments I ran I was unable to get a complete copy of the input.txt on both nodes(N2 and N3) at the same time.
Also tuning the value of tasks.max on nodes (N2 and N3) for the sink connector decided on which node data would be sent.
So, my question is whether I am wrong in expecting such an outcome?
If so then what should I be expecting as a result of the experiment?
If not then how do I get my desired outcome?
Tushar Sudhakar Jee | Software Engineer
c 424.535.8225 | email@example.com
49 Discovery, Suite #220
Irvine, CA 92618