kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Taylor Gautier <tgaut...@tagged.com>
Subject Question - large number of topics
Date Fri, 22 Jul 2011 20:23:20 GMT

I am thinking to use kafka to send/receive messages for a large number of
topics - order of 100k - 1M.

It seems that the directory structure used for topics will probably not work
for this usage.  Also, I'm not sure if the in-memory data structures might
suffer - and also it may be problematic for zookeeper.

One thought I have is to modify the directory structure to be a tree of
directories.  Not sure what if anything might need to be done to in-memory
structures or zookeeper info.

Any thoughts?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message