kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Skøtt <martin.sko...@falconsocial.com>
Subject Coordinator issues with 0.9 consumer
Date Wed, 18 Nov 2015 15:40:05 GMT
Hi,

I'm tracking the 0.9.0.0 Git tag and have a Java consumer using the new
API, but I'm seeing some strange issues. I run ZooKeeper and Kafka on my
own machine using the settings files in config/ and no authentication.
Build is done using Oracle JDK 8. I have 13 topics, each created with a
replication factor of 1.

The first time my consumer connects to the newly created topics (before
anything has been written to the topic) it emits the following errors:

[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version :
0.9.0.0
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId :
fe855f98263cb7b8
[main] ERROR
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto
offset commit failed.
org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException: The
group coordinator is not available.
[main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
- Marking the coordinator 2147483647 dead.
[main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
- Attempt to join group xxx-yyy-reader failed due to obsolete coordinator
information, retrying.
[main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
- Marking the coordinator 2147483647 dead.
[main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
- Attempt to join group xxx-yyy-reader failed due to obsolete coordinator
information, retrying.

If I restart my consumer then it starts without issue.

When I start writing to the topics then my consumer reads data, but will
often log messages like these:

[main] ERROR
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto
offset commit failed.
org.apache.kafka.clients.consumer.internals.SendFailedException

And occasionally these:
[main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator
- Attempt to heart beat failed since member id is not valid, reset it and
try to re-join group.
[main] ERROR
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Error
ILLEGAL_GENERATION occurred while committing offsets for group
audience-indexer-reader
[main] ERROR
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto
offset commit failed.
org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be
completed due to group rebalance
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:497)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:438)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:673)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:646)
at
org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:167)
at
org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:133)
at
org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:107)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(ConsumerNetworkClient.java:350)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:288)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:303)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:197)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:187)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:157)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:352)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.maybeAutoCommitOffsetsSync(ConsumerCoordinator.java:384)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinPrepare(ConsumerCoordinator.java:236)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:209)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.ensurePartitionAssignment(ConsumerCoordinator.java:304)
at
org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:861)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:829)

Does anyone have any input on what might be happening?

-- 
Martin Skøtt

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message