ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jayush Luniya <jlun...@hortonworks.com>
Subject Re: Greetings from China Mobile Developers Focusing on Ambari Community
Date Tue, 23 Oct 2018 17:18:39 GMT
Hi Jie,

The configs defined in the stack definitions are only the default values and can be changed
by the end user when deploying the cluster.
You can follow these steps to change the log4j properties from the UI without having to change
the default.
https://community.hortonworks.com/content/supportkb/49706/kafka-log-file-rotation-based-on-file-size.html

Regards
Jayush

From: "wangjie@cmss.chinamobile.com" <wangjie@cmss.chinamobile.com>
Date: Tuesday, October 23, 2018 at 7:45 AM
To: jluniya <jluniya@apache.org>, atkach <atkach@apache.org>, sumitmohant <sumitmohant@apache.org>,
afernandez <afernandez@apache.org>
Cc: 姚磊 <yaolei@cmss.chinamobile.com>, 王宝晗 <wangbaohan@cmss.chinamobile.com>
Subject: Greetings from China Mobile Developers Focusing on Ambari Community

Dear Ambari PMC:

We are developers from China Mobile and have been contributing a great deal of patches to
the ambari community. We have been using ambari for managing clusters with one of which has
the maximum 1500+ nodes,and we are plannig to apply ambari 2.7 to a cluster with nearly
5000 nodes.

When using ambari to manage the cluster, we found that the nodes  on which had KAFKA service
running were inefficient and some of them even broke down  because the /var disk was full
of kafka logs.

Looking deep into this problem, we found that in ambari the KAFKA service log4j which uses
DailyRollingFileAppender has some properties that it does not support maxBackupIndex and maxFileSize,
which results that the kafka logs’s constantly increasing and would never be removed. Fortunately,
the log4j appender RollingFileAppender supports these properties. Therefore we changed the
entire kafka log4j appender from DailyRollingFileAppender to RollingFileAppender and the problem
was solved.

According to the ambari dev mailing list, we know that the community is going to do a short
follow up maintenance release Apache Ambari 2.7.3, we hope our patch Ambari-20390  would be
contributed to the lastest release version.  And we are looking forward to the release of
ambari 2.7.3 so that we can use it to manage our 5000 nodes cluster.

Best Regards,
Jie Wang

China Moible

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message