kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Nay <maxime...@gmail.com>
Subject Re: Partition reassignment stucked
Date Mon, 09 Dec 2013 20:02:57 GMT
Ok, thanks for your help. When 0.8.1 will be production ready, will you
announce it somewhere (will you release it right away) ?

Thanks,
Maxime



2013/12/9 Neha Narkhede <neha.narkhede@gmail.com>

> I wouldn't call 0.8.1 production ready just yet. We are still in the
> process of deploying it at LinkedIn. Until it is ready, there isn't a good
> cluster expansion solution other than spinning up a new cluster. This is
> probably a little easier if you have a VIP in front of your kafka cluster.
>
> Thanks,
> Neha
>
>
> On Mon, Dec 9, 2013 at 9:40 AM, Maxime Nay <maximenay@gmail.com> wrote:
>
> > Hi,
> >
> > We used the code checked in this branch a few hours before the official
> > 0.8.0 final release : https://github.com/apache/kafka/tree/0.8
> > So hopefully it should be the exact same code as the official release.
> >
> > The controller logs are empty.
> >
> > In a previous exchange you advised us to not use trunk unless we felt
> > adventurous.
> > So if you think 0.8.1 is production ready, and that it shouldn't crash
> > unexpectedly, we can probably give it a try. Otherwise, since our data
> > pipeline heavily relies on Kafka, we might have to use another
> solution...
> > (bring up a brand new cluster each time we want to resize the cluster ?)
> >
> > Thanks
> > Maxime
> >
> >
> >
> > 2013/12/8 Neha Narkhede <neha.narkhede@gmail.com>
> >
> > > Unfortunately quite a few bugs with reassignment are fixed only in
> > 0.8.1. I
> > > wonder if you can run trunk and see how that goes?
> > >
> > > Thanks,
> > > Neha
> > > On Dec 6, 2013 9:46 PM, "Jun Rao" <junrao@gmail.com> wrote:
> > >
> > > > Are you using the 0.8.0 final release? Any error in controller log?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Fri, Dec 6, 2013 at 4:38 PM, Maxime Nay <maximenay@gmail.com>
> > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > We are trying to add a broker to a 10 node cluster. We have 7
> > different
> > > > > topics, each of them is divided in 10 partitions, and their
> > replication
> > > > > factor is 3.
> > > > >
> > > > > To send traffic to this new node, we tried the
> > > > kafka-reassign-partitions.sh
> > > > > tool, but for some reason, it doesnt work, and now it seems that
we
> > are
> > > > > stuck in some kind of partition reassignment process.
> > > > > We can't see any data on our new node, and when we try to execute
> the
> > > > > partition reassignment tool for another topic it says :
> > > > >
> > > > > Partitions reassignment failed due to Partition reassignment
> > currently
> > > in
> > > > > progress for Map([AD_EVENTS,5] ->
> > > > > ReassignedPartitionsContext(List(2),null), [AD_EVENTS,3] ->
> > > > > ReassignedPartitionsContext(List(1),null), [AD_EVENTS,4] ->
> > > > > ReassignedPartitionsContext(List(3),null)). Aborting operation
> > > > > kafka.common.AdminCommandFailedException: Partition reassignment
> > > > currently
> > > > > in progress for Map([AD_EVENTS,5] ->
> > > > > ReassignedPartitionsContext(List(2),null), [AD_EVENTS,3] ->
> > > > > ReassignedPartitionsContext(List(1),null), [AD_EVENTS,4] ->
> > > > > ReassignedPartitionsContext(List(3),null)). Aborting operation
> > > > >     at
> > > > >
> > > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:201)
> > > > >     at
> > > > >
> > > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:140)
> > > > >     at
> > > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> > > > >
> > > > >
> > > > > Any help would be greatly appreciated !
> > > > >
> > > > > Thanks
> > > > > Maxime
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message