kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix GV <fe...@mate1inc.com>
Subject Re: Kafka 155
Date Wed, 11 Apr 2012 21:02:45 GMT
Intra cluster replication is great and would alleviate (or probably
eliminate) the need to have graceful decommission.

But that still does not answer the question: if one had to gracefully
decommission a broker today in 0.7 (or in trunk or w/ patches), how would
one do it?

How can we make a broker read-only?

Wouldn't there be a way to somehow bring consumers down, bring down the
brokers that need to be decommissioned, then re-start those decommissioned
brokers in another network where no producers are pushing any content into
them, then restart the consumers inside of that other network to consume
the stuff they had not consumed yet, and then move (stop and restart) the
consumers back into their original network, where they could see and
consume from the other brokers (which were not decommissioned and were
still receiving stuff from the producers on that network).

That seems awfully convoluted, and I'm not even sure it would work. Plus it
would imply that there is some down time of the consumers (which matters
for real-time oriented consumers, but less so for batch-oriented ones...).
But still, I can't believe there wouldn't be any way at all...

Is there a way to achieve something like that?


On Thu, Apr 5, 2012 at 9:17 PM, Bateman, Matt <mabateman@ebay.com> wrote:

> Hi Jun,
> That would definitely solve the issue. I guess it's just a matter of
> timing for 0.8...
> Thanks,
> Matt
> -----Original Message-----
> From: Jun Rao [mailto:junrao@gmail.com]
> Sent: Thursday, April 05, 2012 6:09 PM
> To: kafka-users@incubator.apache.org
> Subject: Re: Kafka 155
> Matt,
> The main motivation for decommission is that one can let consumers drain
> messages from a broker before taking it out. In 0.8, we are adding intra
> cluster replication. So, a broker can be taken out anytime without
> affecting consumers. Do you still see a need for decommission then?
> Thanks,
> Jun
> On Thu, Apr 5, 2012 at 2:13 PM, Bateman, Matt <mabateman@ebay.com> wrote:
> > Hi Guys,
> >
> > I see that 155 isn't targeted for a release. It deals with gracefully
> > decommissioning a broker. Is there another way to achieve this? In
> > other words, is there a way to prevent publishing to a broker but
> > still allow consumers to pull messages until the broker is "empty"?
> >
> > Thanks,
> >
> > Matt
> >
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message