kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liju John <lijubj...@gmail.com>
Subject Kafka RecordAccumulator size grows beyond configured buffer.memory size
Date Wed, 05 Aug 2015 19:41:41 GMT
Hi,

We are experiencing an issue where the RecordAccumulator size is growing
beyond the configured buffer.memory size .

Below is the sequence of events when this issue occured -

1. One of the broker out of 10 brokers  was brought down at 15:10 PM
2.Till 17:11 PM (same date) the kafka producer instance was in healthy state
3. The same down broker was brought up at ~17:11PM ( same date)
4. Observed UnknownTopicOrPartitionException for some messages
5. HeapDump on  17:14 shows the RecordAccumulator size around ~2.9 GB way
above the configured buffer.size =120MB


Other Details  -

kafka version = 0.8.2.1

Below is the kafka producer properties configured  -

kafka.bootstrap.servers=qa-vip-url
kafka.acks=1
kafka.buffer.memory=125829120
kafka.compression.type=none
kafka.retries=3
kafka.batch.size=200
kafka.client.id=service-instance-id
kafka.linger.ms=0
kafka.max.request.size=5242880
kafka.receive.buffer.bytes=32768
kafka.send.buffer.bytes=131072
kafka.timeout.ms=3000
kafka.block.on.buffer.full=false
kafka.metadata.fetch.timeout.ms=60000
kafka.metadata.max.age.ms=300000
kafka.reconnect.backoff.ms=30
kafka.retry.backoff.ms=3000
kafka.key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
kafka.value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer


Please help in identifing the root cause which is causing this memory leak
for RecordAccumulator.

Attached is the HeapDump overview for reference

Let me know if you need any other information

Regards,
Liju John

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message