nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Geercken" <uwe.geerc...@web.de>
Subject Nifi and Kafka Inserts and Updates
Date Fri, 13 Oct 2017 19:20:32 GMT
<html><head></head><body><div style="font-family: Verdana;font-size:
12.0px;"><div>Hello,</div>

<div>&nbsp;</div>

<div>I am looking for some advice: I have Nifi sending flowfiles to Kafka. As we know
for Kafka everything is an &quot;insert&quot;. Messages are inserted into the Kafka
log.</div>

<div>&nbsp;</div>

<div>Now I wonder what is the best way to insert OR update a relational database table
from Kafka messages using Nifi. What is the best way to determine if the record (data) I get
from Kafka needs to be updated or inserted in the relational db table? Ok. MySQL e.g. has
an &quot;upsert&quot; mode, but I don&#39;t see we have a processor for that.
So I would need to generate the SQL statement and execute that. But is there a more elegant
(straight ahead) way?</div>

<div>&nbsp;</div>

<div>I could use MongoDB as well where we have a processor that allows for an upsert,
but I wanted to know how to solve this with a relational db.</div>

<div>&nbsp;</div>

<div>My idea was to use a db lookup service controller - to lookup if a certain key
exists - but there is none from what I have seen. I don&#39;t think it is an unusual use
case, so I hope that somebody can help or share thought with me.</div>

<div>&nbsp;</div>

<div>Thanks,</div>

<div>&nbsp;</div>

<div>Uwe</div></div></body></html>

Mime
View raw message