kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sachin Mittal <sjmit...@gmail.com>
Subject need some help in understanding weird IllegalStateException: Log end offset should not change while restoring
Date Sun, 18 Dec 2016 07:32:52 GMT
Hi,
What I have noticed recently is that when running the streams application
for a while it exits with some uncaught exception.

The issue was not happening before and I started seeing recently only. Not
sure what we did wrong.
Version is kafka_2.10-0.10.0.1

Is this some bug in kafka. What could be the reason for such an issue.

Thanks
Sachin
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Here is the complete stacktrace

User provided listener
org.apache.kafka.streams.processor.internals.StreamThread$1
for group test failed on partition assignment
java.lang.IllegalStateException: Log end offset should not change while
restoring
        at org.apache.kafka.streams.processor.internals.
ProcessorStateManager.restoreActiveState(ProcessorStateManager.java:257)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
ProcessorStateManager.register(ProcessorStateManager.java:212)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
ProcessorContextImpl.register(ProcessorContextImpl.java:116)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.state.internals.RocksDBWindowStore.init(RocksDBWindowStore.java:184)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.state.internals.MeteredWindowStore.init(MeteredWindowStore.java:66)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.AbstractTask.
initializeStateStores(AbstractTask.java:81) ~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamTask.<init>(StreamTask.java:115) ~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.createStreamTask(StreamThread.java:550)
[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.addStreamTasks(StreamThread.java:577)
[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.access$000(StreamThread.java:68) [kafka-streams-0.10.0.1.jar:
na]
        at org.apache.kafka.streams.processor.internals.StreamThread$1.
onPartitionsAssigned(StreamThread.java:123) ~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.
onJoinComplete(ConsumerCoordinator.java:234) [kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:232)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:227)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.fireSuccess(RequestFuture.java:133)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.complete(RequestFuture.java:107) [kafka-clients-0.10.0.1.jar:
na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture$2.onSuccess(RequestFuture.java:182)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.fireSuccess(RequestFuture.java:133)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.complete(RequestFuture.java:107) [kafka-clients-0.10.0.1.jar:
na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
SyncGroupResponseHandler.handle(AbstractCoordinator.java:436)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
SyncGroupResponseHandler.handle(AbstractCoordinator.java:422)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:679)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:658)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture$1.onSuccess(RequestFuture.java:167)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.fireSuccess(RequestFuture.java:133)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.complete(RequestFuture.java:107) [kafka-clients-0.10.0.1.jar:
na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(
ConsumerNetworkClient.java:426) [kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:278)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:360)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient.poll(ConsumerNetworkClient.java:192)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient.poll(ConsumerNetworkClient.java:163)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.
ensureActiveGroup(AbstractCoordinator.java:243) [kafka-clients-0.10.0.1.jar:
na]
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.
ensurePartitionAssignment(ConsumerCoordinator.java:366)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.
pollOnce(KafkaConsumer.java:978) [kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:938)
[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.runLoop(StreamThread.java:295) [kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.run(StreamThread.java:218) [kafka-streams-0.10.0.1.jar:na]
Uncaught exception at : Sat Dec 17 20:34:38 IST 2016
org.apache.kafka.streams.errors.StreamsException: Failed to rebalance
        at org.apache.kafka.streams.processor.internals.
StreamThread.runLoop(StreamThread.java:299) ~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.run(StreamThread.java:218) ~[kafka-streams-0.10.0.1.jar:na]
Caused by: java.lang.IllegalStateException: Log end offset should not
change while restoring
        at org.apache.kafka.streams.processor.internals.
ProcessorStateManager.restoreActiveState(ProcessorStateManager.java:257)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
ProcessorStateManager.register(ProcessorStateManager.java:212)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
ProcessorContextImpl.register(ProcessorContextImpl.java:116)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.state.internals.RocksDBWindowStore.init(RocksDBWindowStore.java:184)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.state.internals.MeteredWindowStore.init(MeteredWindowStore.java:66)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.AbstractTask.
initializeStateStores(AbstractTask.java:81) ~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamTask.<init>(StreamTask.java:115) ~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.createStreamTask(StreamThread.java:550)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.addStreamTasks(StreamThread.java:577)
~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.access$000(StreamThread.java:68) ~[kafka-streams-0.10.0.1.jar:
na]
        at org.apache.kafka.streams.processor.internals.StreamThread$1.
onPartitionsAssigned(StreamThread.java:123) ~[kafka-streams-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.
onJoinComplete(ConsumerCoordinator.java:234) ~[kafka-clients-0.10.0.1.jar:
na]
        at org.apache.kafka.clients.consumer.internals.
AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:232)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:227)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.fireSuccess(RequestFuture.java:133)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.complete(RequestFuture.java:107) ~[kafka-clients-0.10.0.1.jar:
na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture$2.onSuccess(RequestFuture.java:182)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.fireSuccess(RequestFuture.java:133)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.complete(RequestFuture.java:107) ~[kafka-clients-0.10.0.1.jar:
na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
SyncGroupResponseHandler.handle(AbstractCoordinator.java:436)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
SyncGroupResponseHandler.handle(AbstractCoordinator.java:422)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:679)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:658)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture$1.onSuccess(RequestFuture.java:167)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.fireSuccess(RequestFuture.java:133)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
RequestFuture.complete(RequestFuture.java:107) ~[kafka-clients-0.10.0.1.jar:
na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(
ConsumerNetworkClient.java:426) ~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:278)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:360)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient.poll(ConsumerNetworkClient.java:192)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient.poll(ConsumerNetworkClient.java:163)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.
ensureActiveGroup(AbstractCoordinator.java:243)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.
ensurePartitionAssignment(ConsumerCoordinator.java:366)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.
pollOnce(KafkaConsumer.java:978) ~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:938)
~[kafka-clients-0.10.0.1.jar:na]
        at org.apache.kafka.streams.processor.internals.
StreamThread.runLoop(StreamThread.java:295) ~[kafka-streams-0.10.0.1.jar:na]
        ... 1 common frames omitted

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message