kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Puneet Mehta <mehta.p...@gmail.com>
Subject Max message size & fetch size related?
Date Mon, 24 Sep 2012 22:51:12 GMT
Hi all,

We are using kafka 0.7.1

 I am seeing this error in the producer ->

kafka.common.MessageSizeTooLargeException
at
kafka.producer.SyncProducer$$anonfun$kafka$producer$SyncProducer$$verifyMessageSize$1.apply(SyncProducer.scala:141)
at
kafka.producer.SyncProducer$$anonfun$kafka$producer$SyncProducer$$verifyMessageSize$1.apply(SyncProducer.scala:139)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30)
at kafka.message.MessageSet.foreach(MessageSet.scala:87)
at
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$verifyMessageSize(SyncProducer.scala:139)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at
kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
at kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
at kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
at kafka.producer.Producer.zkSend(Producer.scala:143)
at kafka.producer.Producer.send(Producer.scala:105)
at kafka.javaapi.producer.Producer.send(Producer.scala:104)

We are using the max message size as 1 <1000000>000000 <1000000> bytes.

I am planning to bump this up to say 5 <5000000>000000 <5000000> bytes.

I am just wondering do I need to change any other properties in
producer/broker/consumer that may get impacted by this bump up?

Also, I came across this thread, which relates to fetch size relative to
max message size -> https://issues.apache.org/jira/browse/KAFKA-247

Could anyone of you advise me on places I may get impacted and change
accordingly?


Thanks,
Puneet Mehta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message