kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ran Lupovich <ranlupov...@gmail.com>
Subject Re: JDBC Sink Connector: Counterpart to Dead Letter Queue to keep track of successfully processed records
Date Tue, 27 Apr 2021 21:39:53 GMT
Just a thought , what about jdbc source connector from the database to the
succsess topic?

בתאריך יום ד׳, 28 באפר׳ 2021, 00:19, מאת Florian McKee ‏<
florian.mckee@gmail.com>:

> Hi,
>
>
> I want to ingest messages that are sent by 3rd parties into our system.
>
>
> The ingestion process is as follows:
>
>    - verify message content
>    - forward invalid messages to ingestion_failure topic
>    - persist valid messages in PostgreSQL
>    - forward messages that have been persisted to ingestion_success topic
>
>
> The last point is key: Only forward messages to ingestion_successful if
> they have been persisted in PostgreSQL. Is there any way I can do that with
> a JDBC sink connector?
>
> I'm basically looking for the counterpart of the Dead Letter Queue for
> records that have been processed successfully by the connector.
>
> If that's beyond the scope of connectors: Any recommendations on how to do
> that any other way?
>
>
>
> Thanks
> Florian
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message