kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harald Kirsch <harald.kir...@raytion.com>
Subject Re: Consumer poll - no results
Date Thu, 08 Dec 2016 09:19:54 GMT
auto.offset.reset is honoured if the consumer group has not committed 
offsets yet, or if the offsets expired (I think this is 
offsets.retention.*).

Otherwise the last committed offsets should be read for that group.

Harald.

On 07.12.2016 18:48, Mohit Anchlia wrote:
> Is auto.offset.reset honored just the first time consumer starts and
> polling? In other words everytime consumer starts does it start from the
> beginning even if it has already read those messages?
>
> On Wed, Dec 7, 2016 at 1:43 AM, Harald Kirsch <harald.kirsch@raytion.com>
> wrote:
>
>> Have you defined
>>
>> auto.offset.reset: earliest
>>
>> or otherwise made sure (KafkaConsumer.position()) that the consumer does
>> not just wait for *new* messages to arrive?
>>
>> Harald.
>>
>>
>>
>> On 06.12.2016 20:11, Mohit Anchlia wrote:
>>
>>> I see this message in the logs:
>>>
>>> [2016-12-06 13:54:16,586] INFO [GroupCoordinator 0]: Preparing to
>>> restabilize group DemoConsumer with old generation 3
>>> (kafka.coordinator.GroupCoordinator)
>>>
>>>
>>>
>>> On Tue, Dec 6, 2016 at 10:53 AM, Mohit Anchlia <mohitanchlia@gmail.com>
>>> wrote:
>>>
>>> I have a consumer polling a topic of Kafka 0.10. Even though the topic has
>>>> messages the consumer poll is not fetching the message. The thread dump
>>>> reveals:
>>>>
>>>> "main" #1 prio=5 os_prio=0 tid=0x00007f3ba4008800 nid=0x798 runnable
>>>> [0x00007f3baa6c3000]
>>>>    java.lang.Thread.State: RUNNABLE
>>>>         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>>>>         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
>>>>         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.
>>>> java:93)
>>>>         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
>>>>         - locked <0x00000006c6d1f8b8> (a sun.nio.ch.Util$3)
>>>>         - locked <0x00000006c6d1f8a8> (a java.util.Collections$
>>>> UnmodifiableSet)
>>>>         - locked <0x00000006c6d1f0b8> (a sun.nio.ch.EPollSelectorImpl)
>>>>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
>>>>         at org.apache.kafka.common.network.Selector.select(
>>>> Selector.java:470)
>>>>         at org.apache.kafka.common.network.Selector.poll(
>>>> Selector.java:286)
>>>>         at org.apache.kafka.clients.NetworkClient.poll(
>>>> NetworkClient.java:260)
>>>>         at org.apache.kafka.clients.consumer.internals.
>>>> ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232)
>>>>         - locked <0x00000006c6d1eff8> (a org.apache.kafka.clients.
>>>> consumer.internals.ConsumerNetworkClient)
>>>>         at org.apache.kafka.clients.consumer.KafkaConsumer.
>>>> pollOnce(KafkaConsumer.java:1031)
>>>>
>>>>
>>>
>

Mime
View raw message