kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Virendra Pratap Singh <vpsi...@yahoo-inc.com.INVALID>
Subject Re: Kafka 'reassign-partitions' behavior if the replica does not catches up
Date Tue, 24 Jun 2014 20:32:22 GMT
In process of giving 0.8.1.1 a try.

However I believe the question still holds true.
If the amount of data getting pumped in a partition is such that any new
replica partition can never catch up, then what would the partition
reassignment tool behavior?
If it will be in an infinitely wait mode waiting for the replica to be in
sync, then is there an alternative to cancel the reassignment operation?

Regards,
Virendra
On 6/23/14, 4:49 PM, "Neha Narkhede" <neha.narkhede@gmail.com> wrote:

>However, in my case I realized that the new re-assigned partitions are not
>getting in sync at all. The in-rate in the existing partition is far more
>then the sync rate between earlier and newly reassigned partitions.
>
>This is probably due to a bug. Partition reassignment is stable in
>0.8.1.1.
>I recommend you upgrade and retry.
>
>Thanks
>Neha
>
>
>On Mon, Jun 23, 2014 at 3:24 PM, Virendra Pratap Singh <
>vpsingh@yahoo-inc.com.invalid> wrote:
>
>> I attempted bin/kafka-reassign-partitions.sh for the topics in my
>>cluster.
>> It was automatic reassignment of topics to new brokers.
>>
>> Kafka duly initiated the partition reassignment. I believe kafka
>> controller will do the new leader election once the re-assigned
>>partitions
>> are in sync.
>>
>> However, in my case I realized that the new re-assigned partitions are
>>not
>> getting in sync at all. The in-rate in the existing partition is far
>>more
>> then the sync rate between earlier and newly reassigned partitions.
>>
>> In this scenario, will kafka wait indefinitely for the newly assigned
>> partition to become sync (which will never be).
>>
>> Secondly what are the options to speed up the partition replica sync
>>rate?
>>
>> Running kafka 0.8.0
>>
>> Regards,
>> Virendra
>>
>>
>>
>>


Mime
View raw message