kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From svante karlsson <s...@csi.se>
Subject Re: Adding partitions...
Date Fri, 23 May 2014 12:41:22 GMT
No reshuffeling will take place. And reading messages and put them back in
again will not remove the messages from their "old" partition so the same
message will the exist in more than one partition - eventually to get aged
out of the oldest partion.

If you use partitioning to distribute the load then it's possible to just
leave the old data in the existing partitions and let newer data fill all
the partitions.

If the partitioning means something to your applications then you can
always create a new topic and populate that from the old topic. This will
double your data for some time before you can age out the old topic.

2014-05-23 12:44 GMT+02:00 Niclas Hedhman <niclas@hedhman.org>:

> Hi,
> we are trying to figure out how to layout our topics and partitions. And
> one thing I can't find in the documentation is;
> What happens to data that is sitting in the 'old partitions' when I add a
> new one?
> My gut feeling says that everything remains in the partitions as they are,
> and if we need to have partitions associated with keys, we have to
> implement this re-shuffling ourselves, i.e. read all the messages out and
> put them back in.
> Is it so? And is there any hooks to assist in this?
> Cheers
> --
> Niclas Hedhman, Software Developer
> 河南南路555弄15号1901室。
> http://www.qi4j.org - New Energy for Java
> I live here; http://tinyurl.com/3xugrbk
> I work here; http://tinyurl.com/6a2pl4j
> I relax here; http://tinyurl.com/2cgsug

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message