kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liju John <lijubj...@gmail.com>
Subject Re: Kafka RecordAccumulator size grows beyond configured buffer.memory size
Date Wed, 05 Aug 2015 20:03:18 GMT
Just to add more info   -

Our message size = 1.5MB , so effectively there was no batching as our
batch size is 200.

Is there any case when the RecordAccumulator can grow beyond the configured
buffer.memory ?

Regards,
Liju John

On Wed, Aug 5, 2015 at 12:41 PM, Liju John <lijubjohn@gmail.com> wrote:

> Hi,
>
> We are experiencing an issue where the RecordAccumulator size is growing
> beyond the configured buffer.memory size .
>
> Below is the sequence of events when this issue occured -
>
> 1. One of the broker out of 10 brokers  was brought down at 15:10 PM
> 2.Till 17:11 PM (same date) the kafka producer instance was in healthy
> state
> 3. The same down broker was brought up at ~17:11PM ( same date)
> 4. Observed UnknownTopicOrPartitionException for some messages
> 5. HeapDump on  17:14 shows the RecordAccumulator size around ~2.9 GB way
> above the configured buffer.size =120MB
>
>
> Other Details  -
>
> kafka version = 0.8.2.1
>
> Below is the kafka producer properties configured  -
>
> kafka.bootstrap.servers=qa-vip-url
> kafka.acks=1
> kafka.buffer.memory=125829120
> kafka.compression.type=none
> kafka.retries=3
> kafka.batch.size=200
> kafka.client.id=service-instance-id
> kafka.linger.ms=0
> kafka.max.request.size=5242880
> kafka.receive.buffer.bytes=32768
> kafka.send.buffer.bytes=131072
> kafka.timeout.ms=3000
> kafka.block.on.buffer.full=false
> kafka.metadata.fetch.timeout.ms=60000
> kafka.metadata.max.age.ms=300000
> kafka.reconnect.backoff.ms=30
> kafka.retry.backoff.ms=3000
>
> kafka.key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
>
> kafka.value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
>
>
> Please help in identifing the root cause which is causing this memory leak
> for RecordAccumulator.
>
> Attached is the HeapDump overview for reference
>
> Let me know if you need any other information
>
> Regards,
> Liju John
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message