kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Baumgartner <chris.baumgart...@fujifilm.com>
Subject Best practices for compacting topics with tombstones
Date Wed, 17 Jul 2019 20:11:18 GMT
Hello,

I'm wondering if anyone has advice on configuring compaction. Here is my
scenario:

A producer writes raw data to topic #1. A stream app reads the data from
topic #1, processes it, writes the processed data to topic #2, and then
writes a tombstone record to topic #1.

So, I don't intend for data to be retained very long in topic #1.

Are there any best practices for configuring compaction on topic #1 in this
case? I don't want to keep the data around very long after it has been
processed, but I also don't want to cause performance issues by compacting
too often.

Thanks.

- Chris

-- 
NOTICE:  This message, including any attachments, is only for the use of 
the intended recipient(s) and may contain confidential, sensitive and/or 
privileged information, or information otherwise prohibited from 
dissemination or disclosure by law or regulation, including applicable 
export regulations.  If the reader of this message is not the intended 
recipient, you are hereby notified that any use, disclosure, copying, 
dissemination or distribution of this message or any of its attachments is 
strictly prohibited.  If you received this message in error, please contact 
the sender immediately by reply email and destroy this message, including 
all attachments, and any copies thereof.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message