kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 张学文 <zhan...@asiainfo.com>
Subject Kafka High Level Consumer OOME
Date Mon, 01 Aug 2016 02:52:14 GMT
Hi

Our kafka consumer application has been running for a week without any problems.
But I face to OOME while trying to consume from one topic 100 partitions by 100 consumers
today.
The configurations for the consumers are there:
zookeeper.session.timeout.ms = 10000
zookeeper.sync.time.ms = 200
auto.commit.interval.ms = 1000
zookeeper.connection.timeout.ms=10000
rebalance.backoff.ms=2000
rebalance.max.retries=10

Java heap -Xms2048m -Xmx4096m

We need to consume about 100 million of the amount of real time streaming data per day.
Messages have about 5kb each.
The version of kafka is 0.8.2.

The exception we faced are there : 

Can U give me some suggestion ? We need to resolve it as soon as possible.
Thanks a lot.



祝 工作顺利~

―――――――――――――――――  
张学文
 
亚信科技(南京)有限公司
中国联通客户事业部 产品二部

地址:南京市鼓楼区古平岗4号紫金智梦园B座亚信科技4F
邮编:210013
手机:13645152638
邮箱:zhangxw@asiainfo.com
―――――――――――――――――
Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message