kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Richardson <ch...@chrisrichardson.net>
Subject Re: Kafka for event sourcing architecture
Date Wed, 18 May 2016 05:46:32 GMT
Oli,

Kafka is only a partial solution.
As I describe here (
http://www.slideshare.net/chris.e.richardson/hacksummit-2016-eventdriven-microservices-events-on-the-outside-on-the-inside-and-at-the-core/57)
an event store is a hybrid of a database and a message broker.
It is a database because it provides an API for inserting events for an
entity and retrieving them by the entity's primary key.
It is a message broker because it provides an API for subscribing to events.
Kafka clearly satisfies the latter but not the former.

Just my two cents.

Chris

-- 
Microservices application platform http://eventuate.io

On Tue, May 17, 2016 at 12:18 AM, Olivier Lalonde <olalonde@gmail.com>
wrote:

> Hi all,
>
> I am considering adopting an "event sourcing" architecture for a system I
> am developing and Kafka seems like a good choice of store for events.
>
> For those who aren't aware, this architecture style consists in storing all
> state changes of the system as an ordered log of events and building
> derivative views as needed for easier querying (using a SQL database for
> example). Those views must be completely derived from the event log alone
> so that the log effectively becomes a "single source of truth".
>
> I was wondering if anyone else is using Kafka for that purpose and more
> specifically:
>
> 1) Can Kafka store messages permanently?
>
> 2) Let's say I throw away my derived view and want to re-build it from
> scratch, is it possible to consume messages from a topic from its very
> first message and once it has caught up, listen for new messages like it
> would normally do?
>
> 2) Does it support transactions? Let's say I want to push 3 messages
> atomically but the producer process crashes after sending only 2 messages,
> is it possible to "rollback" the first 2 messages (e.g. "all or nothing"
> semantics)?
>
> 3) Does it support request/response style semantics or can they be
> simulated? My system's primary interface with the outside world is an HTTP
> API so it would be nice if I could publish an event and wait for all the
> internal services which need to process the event to be "done"
> processing before returning a response.
>
> PS: I'm a Node.js/Go developer so when possible please avoid Java centric
> terminology.
>
> Thanks!
>
> - Oli
>
> --
> - Oli
>
> Olivier Lalonde
> http://www.syskall.com <-- connect with me!
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message