kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Hughes <junk...@hotmail.com>
Subject RE: Is it possible to enforce an "unique constraint" through Kafka?
Date Wed, 07 Jan 2015 02:20:40 GMT
Log compaction though allows it to work as a data store quite well for some use cases .  It's
exactly why I started looking hard at Kafka lately.

"The general idea is quite simple. Rather than maintaining only recent 
log entries in the log and throwing away old log segments we maintain 
the most recent entry for each unique key. This ensures that the log 
contains a complete dataset and can be used for reloading key-based 
state."
https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction

> Date: Tue, 6 Jan 2015 16:34:06 -0800
> Subject: Re: Is it possible to enforce an "unique constraint" through Kafka?
> From: wizzat@gmail.com
> To: users@kafka.apache.org
> 
> Kafka is more of a message queue than a data store. You can use it to store
> history of the queue (certainly a powerful use case for disaster recovery),
> but it's still not really a data store.
> 
> From the Kafka website (kafka.apache.org):
> Apache Kafka is a publish-subscribe messaging [queue] rethought as a
> distributed commit log.
> 
> -Mark
> 
> On Tue, Jan 6, 2015 at 3:14 PM, Joseph Pachod <joseph.pachod@gmail.com>
> wrote:
> 
> > Hi
> >
> > Having read a lot about kafka and its use at linkedin, I'm still unsure
> > whether Kafka can be used, with some mindset change for sure, as a general
> > purpose data store.
> >
> > For example, would someone use Kafka to enforce an "unique constraint"?
> >
> > A simple use case is, in the case of linkedin, unicity of users' login.
> >
> > What would be you recommended implementation for such a need?
> >
> > Thanks in advance
> >
> > Best,
> > Joseph
> >
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message