kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lrnop <lr...@protonmail.com>
Subject Kafka use case query
Date Fri, 10 Jul 2015 08:49:16 GMT

We have a number of different scenarios within our company that we are considering Kafka for.

There is one case in particular that has caused debate. The relevant characteristics are:

Very high throughput - 1000's of messages/second.

Very bursty traffic with unpredictable surges in requests from external sources.

Management of long running request/response sessions.

In Kafka terminology the producer that issues a request must also be the consumer for the

Very short message validity - If a response is not received within a few seconds then the
request is invalid.

The consumers of the requests need per-request interaction with a database, introducing further
sources of latency and unpredictable processing rate.

Not everyone is convinced Kafka is a good choice for this particular case. There are others
- such as the backbone for post-processing of TB of transaction records - where everyone agrees
Kafka would be useful.

I'd be very grateful for any feedback.



Sent from [ProtonMail](https://protonmail.ch), encrypted email based in Switzerland.
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message