metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otto Fowler <ottobackwa...@gmail.com>
Subject Re: [DISCUSSION] How to profile storm component taking massive load.
Date Fri, 09 Jun 2017 17:21:18 GMT
https://storm.apache.org/releases/1.0.0/dynamic-worker-profiling.html ?


On June 9, 2017 at 12:15:51, Laurens Vets (laurens@daemon.be) wrote:

Hi list,

I've installed metron (pre-)0.4.0 on CentOS 7 (VM) and when I start all
services, the load is really high (as in 20+ on 4 and 8 CPUs). The same
install on a CentOS 6 VM is fine with a load of 0.5.

When I look at htop, the pid 27867 has 300%+ CPU% which is:

storm 27867 328 3.2 6653844 1056312 ? Sl 11:53 8:52
/usr/jdk64/jdk1.8.0_77/bin/java -server -Xmx768m
-javaagent:/usr/hdp/current/storm-client/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=localhost,port=8650,wireformat31x=true,mode=multicast,config=/usr/hdp/current/storm-client/contrib/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_6700_JVM

-Djava.library.path=/data1/hadoop/storm/supervisor/stormdist/enrichment-1-1496861469/resources/Linux-amd64:/data1/hadoop/storm/supervisor/stormdist/enrichment-1-1496861469/resources:/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib

-Dlogfile.name=worker.log -Dstorm.home=/usr/hdp/2.5.3.0-37/storm
-Dworkers.artifacts=/var/log/storm/workers-artifacts -Dstorm.conf.file=
-Dstorm.options= -Dstorm.log.dir=/var/log/storm
-Djava.io.tmpdir=/data1/hadoop/storm/workers/44e84652-9940-430a-ae10-7ba5b1962a84/tmp

-Dlogging.sensitivity=S3
-Dlog4j.configurationFile=/usr/hdp/2.5.3.0-37/storm/log4j2/worker.xml
-DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector

-Dstorm.id=enrichment-1-1496861469
-Dworker.id=44e84652-9940-430a-ae10-7ba5b1962a84 -Dworker.port=6700 -cp
/usr/hdp/2.5.3.0-37/storm/lib/asm-5.0.3.jar:/usr/hdp/2.5.3.0-37/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.5.3.0-37/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.5.3.0-37/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.5.3.0-37/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.5.3.0-37/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.5.3.0-37/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.5.3.0-37/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.5.3.0-37/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.5.3.0-37/storm/lib/objenesis-2.1.jar:/usr/hdp/2.5.3.0-37/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.5.3.0-37/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.5.3.0-37/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.5.3.0-37/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.5.3.0-37/storm/lib/storm-core-1.0.1.2.5.3.0-37.jar:/usr/hdp/2.5.3.0-37/storm/lib/storm-rename-hack-1.0.1.2.5.3.0-37.jar:/usr/hdp/2.5.3.0-37/storm/lib/zookeeper.jar:/usr/hdp/2.5.3.0-37/storm/lib/ambari-metrics-storm-sink.jar:/usr/hdp/current/storm-supervisor/conf:/data1/hadoop/storm/supervisor/

stormdist/enrichment-1-1496861469/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf

org.apache.storm.daemon.worker enrichment-1-1496861469
9c90251f-54cc-42d7-8c4e-6f3bb2528752 6700
44e84652-9940-430a-ae10-7ba5b1962a84

Any idea what might cause this or how I can further troubleshoot this
component?

Again, same install on CentOS 6, everything's fine...

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message