kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Susheel Kumar <susheel2...@gmail.com>
Subject Re: Kafka as a database/repository question
Date Thu, 15 Dec 2016 18:11:28 GMT
Sorry, I do not have any info on backup and recovery plan at this point of
time.  Please consider both cases (no backup AND back up)

On Thu, Dec 15, 2016 at 1:06 PM, Tauzell, Dave <Dave.Tauzell@surescripts.com
> wrote:

> What is the plan for backup and recovery of the kafka data?
>
> -Dave
>
> -----Original Message-----
> From: Susheel Kumar [mailto:susheel2777@gmail.com]
> Sent: Thursday, December 15, 2016 12:00 PM
> To: users@kafka.apache.org
> Subject: Kafka as a database/repository question
>
> Hello Folks,
>
> I am going thru an existing design where Kafka is planned to be utilised
> in below manner
>
>
>    1. Messages will pushed to Kafka by producers
>    2. There will be updates to existing messages on ongoing basis.  The
>    expectation is that all the updates are consolidated in Kafka and the
>    latest and greatest version/copy is kept
>    3. Consumers will read the messages from Kafka and push to Solr for
>    ingestion purposes
>    4. There will be no purging/removal of messages since it is expected to
>    replay the messages in the future and perform full-re-ingestion.  So
>    messages will be kept in Kafka for indefinite period similar to database
>    where data once stored remains there and can be used later in teh
> future.
>
>
> Do you see any pitfalls / any issue with this design especially wrt to
> storing the messages indefinitely.
>
>
> Thanks,
> Susheel
> This e-mail and any files transmitted with it are confidential, may
> contain sensitive information, and are intended solely for the use of the
> individual or entity to whom they are addressed. If you have received this
> e-mail in error, please notify the sender by reply e-mail immediately and
> destroy all copies of the e-mail and any attachments.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message