kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 杨珏吉 <jueji.y...@gmail.com>
Subject Re: kafka 0.9.0 is busy waiting when no message available
Date Fri, 02 Nov 2018 08:06:00 GMT
It was reported as KAFKA-3159 fixed in 0.9.0.1. Upgrading solves my problem.
https://issues.apache.org/jira/browse/KAFKA-3159?attachmentSortBy=fileName
杨珏吉 <jueji.yang@gmail.com> 于2018年11月1日周四 上午11:38写道:
>
> Hi Kafka experts
>
> I'm using kafka 0.9.0. I made sure that
> Fetch.MinBytes(fetch.min.bytes) is set to be 1 and Fetch.MaxWaitTime
> (fetch.wait.max.ms) is set to be 250 ms. There are no messages in the
> topic but client (I tried golang client `sarama-cluster` and
> `bin/kafka-console-consumer.sh`) keeps fetching and broker responses
> immediately in 1 ms. This spinning cause huge network and cpu
> consumption.
>
> More details:
> 1. No client side error logs or outputs.
> 2. For empty topic, the broker will wait 250 ms as expected.
> 3. After a new message produced and consumed normally, the broker will
> be waiting 250 ms as no more message are produced. But once the broker
> performs "Deleting index", the broker will again
> response immediately with empty message.
> the `server.log` at the time when broker change from waiting 250 ms to
> responding immediately
>
> [2018-10-26 21:14:52,788] INFO [Group Metadata Manager on Broker 1]:
> Removed 0 expired offsets in 0 milliseconds.
> (kafka.coordinator.GroupMetadataManager)
> [2018-10-26 21:15:21,679] INFO Rolled new log segment for 'sync-0' in
> 1 ms. (kafka.log.Log)
> [2018-10-26 21:15:21,679] INFO Scheduling log segment 0 for log sync-0
> for deletion. (kafka.log.Log)
> [2018-10-26 21:15:21,680] INFO Rolled new log segment for 'Sync-0' in
> 0 ms. (kafka.log.Log)
> [2018-10-26 21:15:21,680] INFO Scheduling log segment 83 for log
> Sync-0 for deletion. (kafka.log.Log)
> [2018-10-26 21:16:21,680] INFO Deleting segment 0 from log sync-0.
> (kafka.log.Log)
> [2018-10-26 21:16:21,680] INFO Deleting index
> /data/log/kafka-logs/sync-0/00000000000000000000.index.deleted
> (kafka.log.OffsetIndex)
> [2018-10-26 21:16:21,680] INFO Deleting segment 83 from log Sync-0.
> (kafka.log.Log)
> [2018-10-26 21:16:21,680] INFO Deleting index
> /data/log/kafka-logs/Sync-0/00000000000000000083.index.deleted
> (kafka.log.OffsetIndex)
>
> Does anyone have ever faced this problem?
>
> --
> Best Regards.
>
> Yang Jueji.



-- 
Best Regards.

Yang Jueji.

Mime
View raw message