kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dillian Murphey <crackshotm...@gmail.com>
Subject Re: Is re-partition hitless process?
Date Wed, 25 Nov 2015 00:47:57 GMT
I see, thanks.  Yes, re-assignment. Got my terminology off.

On Tue, Nov 24, 2015 at 4:45 PM, Gwen Shapira <gwen@confluent.io> wrote:

> ah, re-assignment!
>
> If your re-assignment involves moving leaders to other servers, there will
> be a tiny downtime when the new leader election happens.
> Otherwise the main risk is copying the partition over to the new node - it
> can use lots of network and IO. On a busy system we recommend scripting
> this to do very few (maybe one) partition at a time.
>
> On Tue, Nov 24, 2015 at 4:42 PM, Dillian Murphey <crackshotmule@gmail.com>
> wrote:
>
> > Not adding.  Taking some of the partitions from one kafka server and
> > spreading them to another.
> >
> > On Mon, Nov 23, 2015 at 5:40 PM, Gwen Shapira <gwen@confluent.io> wrote:
> >
> > > By re-partition you mean adding partitions to an existing topics?
> > >
> > > There are two things to note in that case:
> > > 1. It is "hitless" because all it does is create new partitions where
> > > future records can go, it does not actually move data around.
> > > 2. You could be "hit" if your consumer code assumes that all data for a
> > > specific key will be in the same partition. Because all future keys may
> > get
> > > assigned to different partitions than what they had before adding
> > > partitions.
> > >
> > > Hope this helps.
> > >
> > > Gwen
> > >
> > > On Mon, Nov 23, 2015 at 9:38 AM, Dillian Murphey <
> > crackshotmule@gmail.com>
> > > wrote:
> > >
> > > > Can I do this on a production system and not have downtime?  I'm
> using
> > > > kafkamanager to make this easier, but it's just running the
> > re-partition
> > > > task.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message