nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alberto Bengoa <albe...@propus.com.br>
Subject CDC like updates on Nifi
Date Tue, 05 Dec 2017 22:23:47 GMT
Hey folks,

I read about Nifi CDC processor for MySQL and other CDC "solutions" with
Nifi found on Google, like these:

https://community.hortonworks.com/idea/53420/apache-nifi-
processor-to-address-cdc-use-cases-for.html
https://community.hortonworks.com/questions/88686/change-
data-capture-using-nifi-1.html
https://community.hortonworks.com/articles/113941/change-
data-capture-cdc-with-apache-nifi-version-1-1.html

I'm trying a different approach to acquire fresh information from tables,
using triggers on source database's tables to write changes to a "changelog
table".

This is done, but my questions are:

Would Nifi be capable to read this tables, transform these data to generate
a SQL equivalent query (insert/update/delete) to send to Hive and/or
Phoenix with current available processors?

Which would be the best / suggested flow?

The objective is to keep tables on the Data Lake as up-to-date as possible
for real time analyses.

Cheers,
Alberto

Mime
View raw message