kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: Partition reassignment stucked
Date Mon, 09 Dec 2013 18:27:41 GMT
I wouldn't call 0.8.1 production ready just yet. We are still in the
process of deploying it at LinkedIn. Until it is ready, there isn't a good
cluster expansion solution other than spinning up a new cluster. This is
probably a little easier if you have a VIP in front of your kafka cluster.

Thanks,
Neha


On Mon, Dec 9, 2013 at 9:40 AM, Maxime Nay <maximenay@gmail.com> wrote:

> Hi,
>
> We used the code checked in this branch a few hours before the official
> 0.8.0 final release : https://github.com/apache/kafka/tree/0.8
> So hopefully it should be the exact same code as the official release.
>
> The controller logs are empty.
>
> In a previous exchange you advised us to not use trunk unless we felt
> adventurous.
> So if you think 0.8.1 is production ready, and that it shouldn't crash
> unexpectedly, we can probably give it a try. Otherwise, since our data
> pipeline heavily relies on Kafka, we might have to use another solution...
> (bring up a brand new cluster each time we want to resize the cluster ?)
>
> Thanks
> Maxime
>
>
>
> 2013/12/8 Neha Narkhede <neha.narkhede@gmail.com>
>
> > Unfortunately quite a few bugs with reassignment are fixed only in
> 0.8.1. I
> > wonder if you can run trunk and see how that goes?
> >
> > Thanks,
> > Neha
> > On Dec 6, 2013 9:46 PM, "Jun Rao" <junrao@gmail.com> wrote:
> >
> > > Are you using the 0.8.0 final release? Any error in controller log?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Fri, Dec 6, 2013 at 4:38 PM, Maxime Nay <maximenay@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > We are trying to add a broker to a 10 node cluster. We have 7
> different
> > > > topics, each of them is divided in 10 partitions, and their
> replication
> > > > factor is 3.
> > > >
> > > > To send traffic to this new node, we tried the
> > > kafka-reassign-partitions.sh
> > > > tool, but for some reason, it doesnt work, and now it seems that we
> are
> > > > stuck in some kind of partition reassignment process.
> > > > We can't see any data on our new node, and when we try to execute the
> > > > partition reassignment tool for another topic it says :
> > > >
> > > > Partitions reassignment failed due to Partition reassignment
> currently
> > in
> > > > progress for Map([AD_EVENTS,5] ->
> > > > ReassignedPartitionsContext(List(2),null), [AD_EVENTS,3] ->
> > > > ReassignedPartitionsContext(List(1),null), [AD_EVENTS,4] ->
> > > > ReassignedPartitionsContext(List(3),null)). Aborting operation
> > > > kafka.common.AdminCommandFailedException: Partition reassignment
> > > currently
> > > > in progress for Map([AD_EVENTS,5] ->
> > > > ReassignedPartitionsContext(List(2),null), [AD_EVENTS,3] ->
> > > > ReassignedPartitionsContext(List(1),null), [AD_EVENTS,4] ->
> > > > ReassignedPartitionsContext(List(3),null)). Aborting operation
> > > >     at
> > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:201)
> > > >     at
> > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:140)
> > > >     at
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> > > >
> > > >
> > > > Any help would be greatly appreciated !
> > > >
> > > > Thanks
> > > > Maxime
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message