kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: Coordinator issues with 0.9 consumer
Date Wed, 18 Nov 2015 18:16:33 GMT
Hello Martin,

Could you paste the consumer config values in this thread as well? And is
the consumer co-located with the broker?

Guozhang

On Wed, Nov 18, 2015 at 7:40 AM, Martin Skøtt <
martin.skoett@falconsocial.com> wrote:

> Hi,
>
> I'm tracking the 0.9.0.0 Git tag and have a Java consumer using the new
> API, but I'm seeing some strange issues. I run ZooKeeper and Kafka on my
> own machine using the settings files in config/ and no authentication.
> Build is done using Oracle JDK 8. I have 13 topics, each created with a
> replication factor of 1.
>
> The first time my consumer connects to the newly created topics (before
> anything has been written to the topic) it emits the following errors:
>
> [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version :
> 0.9.0.0
> [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId :
> fe855f98263cb7b8
> [main] ERROR
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto
> offset commit failed.
> org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException: The
> group coordinator is not available.
> [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> - Marking the coordinator 2147483647 dead.
> [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> - Attempt to join group xxx-yyy-reader failed due to obsolete coordinator
> information, retrying.
> [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> - Marking the coordinator 2147483647 dead.
> [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> - Attempt to join group xxx-yyy-reader failed due to obsolete coordinator
> information, retrying.
>
> If I restart my consumer then it starts without issue.
>
> When I start writing to the topics then my consumer reads data, but will
> often log messages like these:
>
> [main] ERROR
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto
> offset commit failed.
> org.apache.kafka.clients.consumer.internals.SendFailedException
>
> And occasionally these:
> [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> - Attempt to heart beat failed since member id is not valid, reset it and
> try to re-join group.
> [main] ERROR
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Error
> ILLEGAL_GENERATION occurred while committing offsets for group
> audience-indexer-reader
> [main] ERROR
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto
> offset commit failed.
> org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be
> completed due to group rebalance
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:497)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:438)
> at
>
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:673)
> at
>
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:646)
> at
>
> org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:167)
> at
>
> org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:133)
> at
>
> org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:107)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(ConsumerNetworkClient.java:350)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:288)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:303)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:197)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:187)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:157)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:352)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.maybeAutoCommitOffsetsSync(ConsumerCoordinator.java:384)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinPrepare(ConsumerCoordinator.java:236)
> at
>
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:209)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.ensurePartitionAssignment(ConsumerCoordinator.java:304)
> at
>
> org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:861)
> at
>
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:829)
>
> Does anyone have any input on what might be happening?
>
> --
> Martin Skøtt
>



-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message