kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: need some help in understanding weird IllegalStateException: Log end offset should not change while restoring
Date Sun, 18 Dec 2016 21:55:42 GMT
IIRC, his is a know and already fixed bug.

AFAIK, the fix will be part of Kafka 0.10.1.1 -- the vote for the
release is already running, so it should be available soon.


-Matthias



On 12/17/16 11:32 PM, Sachin Mittal wrote:
> Hi,
> What I have noticed recently is that when running the streams application
> for a while it exits with some uncaught exception.
> 
> The issue was not happening before and I started seeing recently only. Not
> sure what we did wrong.
> Version is kafka_2.10-0.10.0.1
> 
> Is this some bug in kafka. What could be the reason for such an issue.
> 
> Thanks
> Sachin
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> Here is the complete stacktrace
> 
> User provided listener
> org.apache.kafka.streams.processor.internals.StreamThread$1
> for group test failed on partition assignment
> java.lang.IllegalStateException: Log end offset should not change while
> restoring
>         at org.apache.kafka.streams.processor.internals.
> ProcessorStateManager.restoreActiveState(ProcessorStateManager.java:257)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> ProcessorStateManager.register(ProcessorStateManager.java:212)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> ProcessorContextImpl.register(ProcessorContextImpl.java:116)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.state.internals.RocksDBWindowStore.init(RocksDBWindowStore.java:184)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.state.internals.MeteredWindowStore.init(MeteredWindowStore.java:66)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.AbstractTask.
> initializeStateStores(AbstractTask.java:81) ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamTask.<init>(StreamTask.java:115) ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.createStreamTask(StreamThread.java:550)
> [kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.addStreamTasks(StreamThread.java:577)
> [kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.access$000(StreamThread.java:68) [kafka-streams-0.10.0.1.jar:
> na]
>         at org.apache.kafka.streams.processor.internals.StreamThread$1.
> onPartitionsAssigned(StreamThread.java:123) ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.
> onJoinComplete(ConsumerCoordinator.java:234) [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:232)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:227)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.fireSuccess(RequestFuture.java:133)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.complete(RequestFuture.java:107) [kafka-clients-0.10.0.1.jar:
> na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture$2.onSuccess(RequestFuture.java:182)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.fireSuccess(RequestFuture.java:133)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.complete(RequestFuture.java:107) [kafka-clients-0.10.0.1.jar:
> na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
> SyncGroupResponseHandler.handle(AbstractCoordinator.java:436)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
> SyncGroupResponseHandler.handle(AbstractCoordinator.java:422)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
> CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:679)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
> CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:658)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture$1.onSuccess(RequestFuture.java:167)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.fireSuccess(RequestFuture.java:133)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.complete(RequestFuture.java:107) [kafka-clients-0.10.0.1.jar:
> na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(
> ConsumerNetworkClient.java:426) [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:278)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:360)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient.poll(ConsumerNetworkClient.java:192)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient.poll(ConsumerNetworkClient.java:163)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.
> ensureActiveGroup(AbstractCoordinator.java:243) [kafka-clients-0.10.0.1.jar:
> na]
>         at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.
> ensurePartitionAssignment(ConsumerCoordinator.java:366)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.KafkaConsumer.
> pollOnce(KafkaConsumer.java:978) [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:938)
> [kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.runLoop(StreamThread.java:295) [kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.run(StreamThread.java:218) [kafka-streams-0.10.0.1.jar:na]
> Uncaught exception at : Sat Dec 17 20:34:38 IST 2016
> org.apache.kafka.streams.errors.StreamsException: Failed to rebalance
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.runLoop(StreamThread.java:299) ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.run(StreamThread.java:218) ~[kafka-streams-0.10.0.1.jar:na]
> Caused by: java.lang.IllegalStateException: Log end offset should not
> change while restoring
>         at org.apache.kafka.streams.processor.internals.
> ProcessorStateManager.restoreActiveState(ProcessorStateManager.java:257)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> ProcessorStateManager.register(ProcessorStateManager.java:212)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> ProcessorContextImpl.register(ProcessorContextImpl.java:116)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.state.internals.RocksDBWindowStore.init(RocksDBWindowStore.java:184)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.state.internals.MeteredWindowStore.init(MeteredWindowStore.java:66)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.AbstractTask.
> initializeStateStores(AbstractTask.java:81) ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamTask.<init>(StreamTask.java:115) ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.createStreamTask(StreamThread.java:550)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.addStreamTasks(StreamThread.java:577)
> ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.access$000(StreamThread.java:68) ~[kafka-streams-0.10.0.1.jar:
> na]
>         at org.apache.kafka.streams.processor.internals.StreamThread$1.
> onPartitionsAssigned(StreamThread.java:123) ~[kafka-streams-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.
> onJoinComplete(ConsumerCoordinator.java:234) ~[kafka-clients-0.10.0.1.jar:
> na]
>         at org.apache.kafka.clients.consumer.internals.
> AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:232)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:227)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.fireSuccess(RequestFuture.java:133)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.complete(RequestFuture.java:107) ~[kafka-clients-0.10.0.1.jar:
> na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture$2.onSuccess(RequestFuture.java:182)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.fireSuccess(RequestFuture.java:133)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.complete(RequestFuture.java:107) ~[kafka-clients-0.10.0.1.jar:
> na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
> SyncGroupResponseHandler.handle(AbstractCoordinator.java:436)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
> SyncGroupResponseHandler.handle(AbstractCoordinator.java:422)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
> CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:679)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$
> CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:658)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture$1.onSuccess(RequestFuture.java:167)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.fireSuccess(RequestFuture.java:133)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> RequestFuture.complete(RequestFuture.java:107) ~[kafka-clients-0.10.0.1.jar:
> na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(
> ConsumerNetworkClient.java:426) ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:278)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:360)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient.poll(ConsumerNetworkClient.java:192)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.
> ConsumerNetworkClient.poll(ConsumerNetworkClient.java:163)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.
> ensureActiveGroup(AbstractCoordinator.java:243)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.
> ensurePartitionAssignment(ConsumerCoordinator.java:366)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.KafkaConsumer.
> pollOnce(KafkaConsumer.java:978) ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:938)
> ~[kafka-clients-0.10.0.1.jar:na]
>         at org.apache.kafka.streams.processor.internals.
> StreamThread.runLoop(StreamThread.java:295) ~[kafka-streams-0.10.0.1.jar:na]
>         ... 1 common frames omitted
> 


Mime
View raw message