kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tao xiao <xiaotao...@gmail.com>
Subject Re: Kafka RecordAccumulator size grows beyond configured buffer.memory size
Date Thu, 06 Aug 2015 01:41:48 GMT
It may be possibly caused by this issue
https://issues.apache.org/jira/browse/KAFKA-2281. Can you apply the patch
and try again?

On Thu, 6 Aug 2015 at 04:03 Liju John <lijubjohn@gmail.com> wrote:

> Just to add more info   -
>
> Our message size = 1.5MB , so effectively there was no batching as our
> batch size is 200.
>
> Is there any case when the RecordAccumulator can grow beyond the configured
> buffer.memory ?
>
> Regards,
> Liju John
>
> On Wed, Aug 5, 2015 at 12:41 PM, Liju John <lijubjohn@gmail.com> wrote:
>
> > Hi,
> >
> > We are experiencing an issue where the RecordAccumulator size is growing
> > beyond the configured buffer.memory size .
> >
> > Below is the sequence of events when this issue occured -
> >
> > 1. One of the broker out of 10 brokers  was brought down at 15:10 PM
> > 2.Till 17:11 PM (same date) the kafka producer instance was in healthy
> > state
> > 3. The same down broker was brought up at ~17:11PM ( same date)
> > 4. Observed UnknownTopicOrPartitionException for some messages
> > 5. HeapDump on  17:14 shows the RecordAccumulator size around ~2.9 GB way
> > above the configured buffer.size =120MB
> >
> >
> > Other Details  -
> >
> > kafka version = 0.8.2.1
> >
> > Below is the kafka producer properties configured  -
> >
> > kafka.bootstrap.servers=qa-vip-url
> > kafka.acks=1
> > kafka.buffer.memory=125829120
> > kafka.compression.type=none
> > kafka.retries=3
> > kafka.batch.size=200
> > kafka.client.id=service-instance-id
> > kafka.linger.ms=0
> > kafka.max.request.size=5242880
> > kafka.receive.buffer.bytes=32768
> > kafka.send.buffer.bytes=131072
> > kafka.timeout.ms=3000
> > kafka.block.on.buffer.full=false
> > kafka.metadata.fetch.timeout.ms=60000
> > kafka.metadata.max.age.ms=300000
> > kafka.reconnect.backoff.ms=30
> > kafka.retry.backoff.ms=3000
> >
> >
> kafka.key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
> >
> >
> kafka.value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
> >
> >
> > Please help in identifing the root cause which is causing this memory
> leak
> > for RecordAccumulator.
> >
> > Attached is the HeapDump overview for reference
> >
> > Let me know if you need any other information
> >
> > Regards,
> > Liju John
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message