kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrienne Kole <adrienneko...@gmail.com>
Subject Kafka Streams dynamic partitioning
Date Tue, 04 Oct 2016 22:00:21 GMT
Hi,

>From Streams documentation, I can see that each Streams instance is
processing data independently (from other instances), reads from topic
partition(s) and writes to specified topic.


So here, the partitions of topic should be determined beforehand and should
remain static.
In my usecase I want to create partitioned/keyed (time) windows and
aggregate them.
I can partition the incoming data to specified topic's partitions and each
Stream instance can do windowed aggregations.

However, if I don't know the number of possible keys (to partition), then
what should I do?

Thanks
Adrienne

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message