kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Nay <maxime...@gmail.com>
Subject Re: Partition reassignment stucked
Date Tue, 10 Dec 2013 00:23:47 GMT
That's good to know.
Thanks for your help!


2013/12/9 Neha Narkhede <neha.narkhede@gmail.com>

> We will announce it on this mailing list. It is probably a month away from
> a release.
>
> Thanks,
> Neha
>
>
> On Mon, Dec 9, 2013 at 12:02 PM, Maxime Nay <maximenay@gmail.com> wrote:
>
> > Ok, thanks for your help. When 0.8.1 will be production ready, will you
> > announce it somewhere (will you release it right away) ?
> >
> > Thanks,
> > Maxime
> >
> >
> >
> > 2013/12/9 Neha Narkhede <neha.narkhede@gmail.com>
> >
> > > I wouldn't call 0.8.1 production ready just yet. We are still in the
> > > process of deploying it at LinkedIn. Until it is ready, there isn't a
> > good
> > > cluster expansion solution other than spinning up a new cluster. This
> is
> > > probably a little easier if you have a VIP in front of your kafka
> > cluster.
> > >
> > > Thanks,
> > > Neha
> > >
> > >
> > > On Mon, Dec 9, 2013 at 9:40 AM, Maxime Nay <maximenay@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > We used the code checked in this branch a few hours before the
> official
> > > > 0.8.0 final release : https://github.com/apache/kafka/tree/0.8
> > > > So hopefully it should be the exact same code as the official
> release.
> > > >
> > > > The controller logs are empty.
> > > >
> > > > In a previous exchange you advised us to not use trunk unless we felt
> > > > adventurous.
> > > > So if you think 0.8.1 is production ready, and that it shouldn't
> crash
> > > > unexpectedly, we can probably give it a try. Otherwise, since our
> data
> > > > pipeline heavily relies on Kafka, we might have to use another
> > > solution...
> > > > (bring up a brand new cluster each time we want to resize the cluster
> > ?)
> > > >
> > > > Thanks
> > > > Maxime
> > > >
> > > >
> > > >
> > > > 2013/12/8 Neha Narkhede <neha.narkhede@gmail.com>
> > > >
> > > > > Unfortunately quite a few bugs with reassignment are fixed only in
> > > > 0.8.1. I
> > > > > wonder if you can run trunk and see how that goes?
> > > > >
> > > > > Thanks,
> > > > > Neha
> > > > > On Dec 6, 2013 9:46 PM, "Jun Rao" <junrao@gmail.com> wrote:
> > > > >
> > > > > > Are you using the 0.8.0 final release? Any error in controller
> log?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > > On Fri, Dec 6, 2013 at 4:38 PM, Maxime Nay <maximenay@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > We are trying to add a broker to a 10 node cluster. We
have 7
> > > > different
> > > > > > > topics, each of them is divided in 10 partitions, and their
> > > > replication
> > > > > > > factor is 3.
> > > > > > >
> > > > > > > To send traffic to this new node, we tried the
> > > > > > kafka-reassign-partitions.sh
> > > > > > > tool, but for some reason, it doesnt work, and now it seems
> that
> > we
> > > > are
> > > > > > > stuck in some kind of partition reassignment process.
> > > > > > > We can't see any data on our new node, and when we try
to
> execute
> > > the
> > > > > > > partition reassignment tool for another topic it says :
> > > > > > >
> > > > > > > Partitions reassignment failed due to Partition reassignment
> > > > currently
> > > > > in
> > > > > > > progress for Map([AD_EVENTS,5] ->
> > > > > > > ReassignedPartitionsContext(List(2),null), [AD_EVENTS,3]
->
> > > > > > > ReassignedPartitionsContext(List(1),null), [AD_EVENTS,4]
->
> > > > > > > ReassignedPartitionsContext(List(3),null)). Aborting operation
> > > > > > > kafka.common.AdminCommandFailedException: Partition
> reassignment
> > > > > > currently
> > > > > > > in progress for Map([AD_EVENTS,5] ->
> > > > > > > ReassignedPartitionsContext(List(2),null), [AD_EVENTS,3]
->
> > > > > > > ReassignedPartitionsContext(List(1),null), [AD_EVENTS,4]
->
> > > > > > > ReassignedPartitionsContext(List(3),null)). Aborting operation
> > > > > > >     at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:201)
> > > > > > >     at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:140)
> > > > > > >     at
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> > > > > > >
> > > > > > >
> > > > > > > Any help would be greatly appreciated !
> > > > > > >
> > > > > > > Thanks
> > > > > > > Maxime
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message