kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: Consumer hangs at poll() and never throw exception on client
Date Tue, 10 Dec 2019 08:41:59 GMT
Hard to say why the second consumer does not fetch data.

I would try to inspect the logs at DEBUG level to get more insight what
going on.


-Matthias

On 12/10/19 12:27 AM, Frank Zhou wrote:
> "earliest",  as we need everything from the start
> 
> On Tue, Dec 10, 2019 at 4:22 PM Matthias J. Sax <matthias@confluent.io>
> wrote:
> 
>> What is your `auto.offset.reset` policy?
>>
>> On 12/10/19 12:02 AM, Frank Zhou wrote:
>>> Hi,
>>>
>>> Thanks for your prompt reply. Consumers have different group.id in our
>>> case, as we just need all the data in the topic, so every time the
>>> application starts, the consumer will be in a new group. For current
>>> testing environment, we just one partition for one topic. though we have
>>> tried with multiple partitions, and situation happened as well.
>>>
>>> On Tue, Dec 10, 2019 at 3:56 PM Matthias J. Sax <matthias@confluent.io>
>>> wrote:
>>>
>>>> Are both consumers in the same consumer group, ie, use the same `
>> group.id
>>>> `?
>>>>
>>>> If yes, how many partitions does the topic have? If it has only one
>>>> partition, the observed behavior is expected, because a single
>>>> partitions can only be read by a single consumer instance per consumer
>>>> group. The second consumer won't have any assigned partitions to read
>> from.
>>>>
>>>> -Matthias
>>>>
>>>> On 12/9/19 11:12 PM, Frank Zhou wrote:
>>>>> Hi,
>>>>>
>>>>> I am quite new to Kafka, and I have encountered a weird case during QA
>>>>> stage for our application. Now we have 2 consumers consuming same topic
>>>>> in kafka cluster. The first started consumer works fine and get closed
>>>>> after getting all the messages. After that, the second one started and
>>>>> just hang there in the poll(), which we set the Duration to 10 seconds,
>>>>> and will quit the loop if no records are fetched. From kafka server
>> log,
>>>>> I don't see warning or error for the second consumer.
>>>>>
>>>>> Attached client/server log related to both consumers. I am using 2.3.1
>>>>> client, 2.2.2 kafka servers. Looking forward to you feedback.
>>>>>
>>>>>
>>>>> Best regards,
>>>>> --
>>>>> *Frank Zhou*
>>>>> R&D, Itiviti
>>>>> Java Developer
>>>>> D +852 2521 7480
>>>>> frank.zhou@itiviti.com <mailto:frank.zhou@itiviti.com>
>>>>>
>>>>> ______________________________
>>>>>
>>>>> itiviti.com <https://www.itiviti.com/>
>>>>>
>>>>> /The information contained in or attached to this email is strictly
>>>>> confidential. If you are not the intended recipient, please notify us
>>>>> immediately by telephone and return the message to us./
>>>>>
>>>>> /Email communications by definition contain personal information. The
>>>>> Itiviti group of companies is subject to European data protection
>>>>> regulations. Itiviti’s Privacy Notice is available at www.itiviti.com
>>>>> <http://www.itiviti.com/>. Itiviti expects the recipient of this
email
>>>>> to be compliant with Itiviti’s Privacy Notice and applicable
>>>>> regulations. Please advise us immediately at
>>>>> dataprotectionteam@Itiviti.com if you are not compliant with these./
>>>>>
>>>>>
>>>>> ______________________________
>>>>>
>>>>> itiviti.com <https://www.itiviti.com/>
>>>>> <https://www.linkedin.com/company/itiviti> ^Follow Itiviti on Linkedin
>>>>> <https://www.linkedin.com/company/10438325/>
>>>>>
>>>>> The information contained in or attached to this email is strictly
>>>>> confidential. If you are not the intended recipient, please notify us
>>>>> immediately by telephone and return the message to us. Email
>>>>> communications by definition contain personal information. The Itiviti
>>>>> group of companies is subject to European data protection regulations.
>>>>>
>>>>> Itiviti’s Privacy Notice is available at www.itiviti.com
>>>>> <http://www.itiviti.com/>. Itiviti expects the recipient of this
email
>>>>> to be compliant with Itiviti’s Privacy Notice and applicable
>>>>> regulations. Please advise us immediately at
>>>>> dataprotectionteam@Itiviti.com if you are not compliant with these.
>>>>>
>>>>
>>>>
>>>
>>
>>
> 


Mime
View raw message