kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaurav Bajaj <gauravhba...@gmail.com>
Subject Relation between "fetch.max.bytes" and "max.partition.fetch.bytes"
Date Tue, 13 Aug 2019 10:34:23 GMT
Hello Kafka lovers,

I have one doubt regarding these 2 configs in kafka 1.3.0 consumer , 1-
fetch.max.bytes(default 50MB) and 2- max.partition.fetch.bytes(default 1MB).
Objective is to pull 500MB of data in single poll from Java consumer of
Kafka.
Problem is my consumer is getting stuck(not polling anything no exception)
when I set both configs to 600MB.
When I make first config to 60MB, it works fine (I start getting 500MB)

1. Why does it getis stuck and doesn't return any messages in cast when
both properties have 600MB value?
2. What is the relation of these properties with with each other?
3. How come after setting fetch.max.bytes to 60MB, I can still get bigger
batch of size 500MB?


Best Regards,
Gaurav

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message