kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apurva Sharma <apurva.sharm...@gmail.com>
Subject Re: Partition creation issues (0.9.0.1)
Date Mon, 19 Sep 2016 15:46:16 GMT
Bumping for visibility - appreciate any responses on this.

Thanks,
Apurva

On Thu, Sep 15, 2016 at 3:57 PM, Apurva Sharma <apurva.sharma56@gmail.com>
wrote:

>
> Kafka Topic Creation issues (via Kafka-Manager with
> auto.create.topics.enable = false)
> Version: 0.9.0.1
>
> We created a topic "web" via Kafka-Manager (our brokers are configured for
> autocreate to be false) and then clicked on Generate Partitions and
> according to the tool, the topic has been created cleanly with partitions
> assigned correctly to brokers.
> However, when we look into the individual broker logs:
>
> [2016-09-15 18:46:10,268] ERROR [ReplicaFetcherThread-5-1006], Error for
> partition [web,2] to broker 1006:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,391] ERROR [ReplicaFetcherThread-4-1005], Error for
> partition [web,1] to broker 1005:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,391] WARN [Replica Manager on Broker 1001]: While
> recording the replica LEO, the partition [web,5] hasn't been created.
> (kafka.server.ReplicaManager)
> [2016-09-15 18:46:10,407] ERROR [ReplicaFetcherThread-4-1004], Error for
> partition [web,16] to broker 1004:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,472] ERROR [ReplicaFetcherThread-11-1003], Error for
> partition [web,23] to broker 1003:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,520] WARN [Replica Manager on Broker 1001]: While
> recording the replica LEO, the partition [web,29] hasn't been created.
> (kafka.server.ReplicaManager)
> [2016-09-15 18:46:10,667] ERROR [ReplicaFetcherThread-11-1004], Error for
> partition [web,8] to broker 1004:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,895] WARN [Replica Manager on Broker 1001]: While
> recording the replica LEO, the partition [web,21] hasn't been created.
> (kafka.server.ReplicaManager)
> [2016-09-15 18:46:10,931] WARN [Replica Manager on Broker 1001]: While
> recording the replica LEO, the partition [web,13] hasn't been created.
> (kafka.server.ReplicaManager)
>
> Indeed, when we actually inspect if the partitions were actually created,
> we see disparity with what's reported by the tool. (Many partitions are
> actually not present on the brokers yet - leading me to believe that the
> partitioning process is stuck)
>
>
> | ZK Id |         Broker Id          |    According to Kafka Manager    |
>        Actual Partitions Created         |
> |---------|----------------------------|--------------------
> ------------------------|------------------------------------------------|
> |  1001 | broker-1001.xxxx.xxxx.xxxx | 1,2,5,8,9,13,16,21,23,29,30,31   |
> web-1,web-16,web-2,web-23,web-8,web-9    |
> |  1002 | broker-1004.xxxx.xxxx.xxxx | 2,3,6,9,10,14,16,17,22,24,30,31  |
> web-10,web-16,web-3,web-9                |
> |  1003 | broker-1002.xxxx.xxxx.xxxx | 3,4,7,10,11,15,17,18,23,24,25,31 |
> web-10,web-11,web-3,web-4                |
> |  1004 | broker-1003.xxxx.xxxx.xxxx | 0,4,5,8,11,12,16,18,19,24,25,26  |
> web-12,web-18,web-19,web-25,web-26       |
> |  1005 | broker-1005.xxxx.xxxx.xxxx | 1,5,6,9,12,13,17,19,20,25,26,27  |
> web-12,web-13,web-19,web-26,web-27,web-5 |
> |  1006 | broker-1006.xxxx.xxxx.xxxx | 2,6,7,10,13,14,18,20,21,26,27,28 |
> web-27                                   |
> |  1007 | broker-1007.xxxx.xxxx.xxxx | 0,3,7,11,14,15,19,21,22,27,28,29 |
> web-21,web-22,web-7                      |
> |  1008 | broker-1008.xxxx.xxxx.xxxx | 0,1,4,8,12,15,20,22,23,28,29,30  |
> web-1,web-15,web-22,web-29,web-8         |
>
> Finally, when we try to repartition the topic we get:
> "Yikes! Partition reassignment currently in progress for. Aborting
> operation" (This seems expected because the initial partitioning has not
> completed for 3 days - so clearly it's stuck somewhere).
>
> We have created topics with the Kafka-Manager tool before without issues,
> but this has started happening consistently for the past 2-3 topics that we
> tried creating through the tool.
> Any help on this is greatly appreciated.
>
> --
> Regards,
> Apurva
>



-- 
Regards,
Apurva

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message