kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Upendra Yadav <upendra1...@gmail.com>
Subject Re: Producer slowness and Exception on broker restart - kafka.common.KafkaException: Unknown group metadata version 1
Date Thu, 28 Nov 2019 05:47:31 GMT
Hi,

We also did version upgrade and then degraded to old due to some issue.
But that we done ~1 year before.
Do I need to follow this to solve this issue?

http://mail-archives.apache.org/mod_mbox/kafka-dev/201907.mbox/%3CJIRA.13246826.1563975667000.40318.1563975720170@Atlassian.JIRA%3E

In this he follow below steps to evicted the broken records with
temporarily setting the segment time to a very low value and deactivation
of compaction.

/opt/kafka/bin/kafka-topics.sh --alter --config segment.ms=900000
--topic __consumer_offsets
--zookeeper localhost:2181
/opt/kafka/bin/kafka-topics.sh --alter --config cleanup.policy=delete
--topic __consumer_offsets
--zookeeper localhost:2181
< wait for the cleaner to clean up >
/opt/kafka/bin/kafka-topics.sh --alter --config segment.ms=604800000
--topic __consumer_offsets
--zookeeper localhost:2181
/opt/kafka/bin/kafka-topics.sh --alter --config cleanup.policy=compact
--topic __consumer_offsets
--zookeeper localhost:2181{code}




On Wed, Nov 27, 2019 at 10:33 PM Upendra Yadav <upendra1024@gmail.com>
wrote:

> Hi Team,
>
> In my production setup, I’m getting below exception(for 4-5
> __consumer_offsets partitions) while restarting kafka broker(s).
> Before restart, my producers(sync) are very slow. for some messages its
> taking 5 seconds to acknowledge.
> But after restart, all messages wishing 2 milliseconds.
> After sometime once again same producer slowness.
> Before restart was able to produce 7k messages, but after restart able to
> push 1lakh messages in 10 minutes.
>
> Kafka broker and client version- 10.0.0.1
> Using latest producer java client for this version.
> But consumers are old scala api.
>
> Exception:
>
> [2019-11-27 05:54:43,724] INFO [Group Metadata Manager on Broker 3]:
> Loading offsets and group metadata from [__consumer_offsets,34]
> (kafka.coordinator.GroupMetadataManager)
> [2019-11-27 05:54:43,735] ERROR [Group Metadata Manager on Broker 3]:
> Error in loading offsets from [__consumer_offsets,34]
> (kafka.coordinator.GroupMetadataManager)
> kafka.common.KafkaException: Unknown group metadata version 1
>         at
> kafka.coordinator.GroupMetadataManager$.schemaForGroup(GroupMetadataManager.scala:764)
>         at
> kafka.coordinator.GroupMetadataManager$.readGroupMessageValue(GroupMetadataManager.scala:934)
>         at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:412)
>         at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:383)
>         at scala.collection.Iterator$class.foreach(Iterator.scala:893)
>         at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30)
>         at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>         at kafka.message.MessageSet.foreach(MessageSet.scala:71)
>         at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply$mcV$sp(GroupMetadataManager.scala:383)
>         at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
>         at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
>         at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)
>         at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:239)
>         at
> kafka.coordinator.GroupMetadataManager.kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1(GroupMetadataManager.scala:374)
>         at
> kafka.coordinator.GroupMetadataManager$$anonfun$loadGroupsForPartition$1.apply$mcV$sp(GroupMetadataManager.scala:353)
>         at
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)
>         at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:56)
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message