kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maria Pilar <pilife...@gmail.com>
Subject Capturing and storing these Kafka events for query.
Date Thu, 11 Jan 2018 16:55:41 GMT
Hi all,

I have a requirement to be able to capture and store events for query,
and I'm trying to choose the best option for that:

1) Capture the events from a separate topic, store events a state, in
order to convert a stream to a table, that means materializing the
stream.
The option for it, that I'm thinking is to use an external state that
is maintained in an external datastore, often a NoSQL (e.g Cassandra).
That solution provides the advantage unlimited size and it can be
accessed from multiple instances of the application or from different
applications.

2) Other options could be query direct in local o internal state or in
memory, however, to be able to query directly onto a topic, I would
need to search a record by partition and offset, I don't know exactly
how to implement that option if it's possible.

Cheers.

Mime
View raw message