storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brad Rhodes <bradr...@hotmail.com>
Subject RE: Nimbus setting java.class.path to include javaagent
Date Wed, 05 Oct 2016 21:14:37 GMT
Hey Taylor,Sorry, that path was from my development machine.
On the nimbus server in question storm.classpath returns:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/clj-stacktrace-0.2.2.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/ring-devel-0.3.11.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/logback-core-1.0.13.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.6.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/disruptor-2.10.4.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/slf4j-api-1.7.5.jar:/opt/storm/lib/logback-classic-1.0.13.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/ring-core-1.1.5.jar
From: bradrhod@hotmail.com
To: user@storm.apache.org
Subject: RE: Nimbus setting java.class.path to include javaagent
Date: Wed, 5 Oct 2016 12:52:20 -0700




That may be the case.  
I have confirmed that supervisor is getting either the Client classpath or Server classpath
sent to zookeeper to start worker processes.  If I edit the config file on the supervisor,
the workers still start with the classpath that is on the nimbus server, with the javaagent
included in the classpath.  The only way that seems possible is that supervisor is starting
with a classpath it has gotten from zookeeper.
Yes, the weird part is that the agent is sneaking onto the class path when the worker starts
up.
storm classpath returns: /usr/local/Cellar/storm/1.0.1/libexec/lib/asm-5.0.3.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/clojure-1.7.0.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/disruptor-3.3.2.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/kryo-3.0.3.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/log4j-api-2.1.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/log4j-core-2.1.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/log4j-slf4j-impl-2.1.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/minlog-1.3.0.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/objenesis-2.1.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/reflectasm-1.10.1.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/servlet-api-2.5.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/slf4j-api-1.7.7.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/storm-core-1.0.1.jar:/usr/local/Cellar/storm/1.0.1/libexec/lib/storm-rename-hack-1.0.1.jar
thanks for all of your help.brad

Subject: Re: Nimbus setting java.class.path to include javaagent
From: ptgoetz@gmail.com
Date: Wed, 5 Oct 2016 15:33:47 -0400
To: user@storm.apache.org

What your seeing is the ZooKeeper client running within Nimbus/Supervisor just printing out
it’s class path, which it inherits from Storm’s class path. It’s not really sending
that to ZooKeeper.
What’s weird is that the agent jar sneaking onto the class path.
On your supervisor node, can you run `storm classpath` and see what it prints?
-Taylor

On Oct 5, 2016, at 1:17 PM, Brad Rhodes <bradrhod@hotmail.com> wrote:Storm 0.9.6.  We
are working on moving to 1.0.2 as we speak. nimbus.childopts: "-Xmx1024m -DjmxRegistryPort=1098
-DjmxServerPort=1099 -javaagent:/opt/monitoring_agent/bobbrad.jar -agentlib:jdwp=transport=dt_socket,server=y,address=8100,suspend=
y -Dmonitoring.agent.properties=/opt/monitoring_agent/monitoring.properties -Dapplication.name=storm.test.lodgingamenity.nimbus"


supervisor.childopts: "-Xmx256m -DjmxRegistryPort=1098 -DjmxServerPort=1099 -javaagent:/opt/monitoring_agent/bobbrad2.jar
-Dmonitoring.agent.properties=/opt/monitoring_agent/monitoring
.properties -Dapplication.name=storm.test.lodgingamenity.supervisor"


worker.childopts: "-Xmx1024m -Dmonitoring.agent.properties=/opt/monitoring_agent/monitoring.properties
-Dapplication.name=storm.test.lodgingamenity.worker.%ID%"  From: P. Taylor Goetz [mailto:ptgoetz@gmail.com]

Sent: Tuesday, October 4, 2016 9:46 AM
To: user@storm.apache.org
Subject: Re: Nimbus setting java.class.path to include javaagent Can you post your values
for nimbus.childopts, supervisor.childopts, and worker.childopts? Also what version of Storm
are you on? -Taylor On Sep 28, 2016, at 6:06 PM, Brad Rhodes <bradrhod@hotmail.com>
wrote: I have a situation where we have a javaagent jar file.  Someone has added a number
of classes to the corporate standard javaagent jar file.  Nimbus appears to append -javaagent
jar files to the -cp (claspath) for the supervisor process. When Nimbus starts up java is
started with a command like:  /usr/java/latest/bin/java -server -Dstorm.options= -Dstorm.home=/opt/storm
-Dstorm.log.dir=/opt/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
-Dstorm.conf.file= -cp /opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/clj-stacktrace-0.2.2.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/ring-devel-0.3.11.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/logback-core-1.0.13.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.6.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/disruptor-2.10.4.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/slf4j-api-1.7.5.jar:/opt/storm/lib/logback-classic-1.0.13.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/conf
-Xmx1024m -DjmxRegistryPort=1098 -DjmxServerPort=1099 -javaagent:/opt/monitoring_agent/bobbrad.jar
-agentlib:jdwp=transport=dt_socket,server=y,address=8100,suspend=y -Dmonitoring.agent.properties=/opt/monitoring_agent/monitoring.properties
-Dapplication.name=storm.test.lodgingamenity.nimbus -Dlogfile.name=nimbus.log -Dlogback.configurationFile=/opt/storm/logback/cluster.xml
backtype.storm.daemon.nimbus Nimbus at some point sends the following to Zookeeper:016-09-28T18:50:45.627+0000
o.a.s.z.ZooKeeper [INFO] Client environment:java.class.path=/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/clj-stacktrace-0.2.2.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/ring-devel-0.3.11.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/logback-core-1.0.13.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.6.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/disruptor-2.10.4.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/slf4j-api-1.7.5.jar:/opt/storm/lib/logback-classic-1.0.13.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/conf:/opt/monitoring_agent/bobbrad.jar
and Server environment:java.class.path=/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/clj-stacktrace-0.2.2.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/ring-devel-0.3.11.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/logback-core-1.0.13.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.6.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/disruptor-2.10.4.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/slf4j-api-1.7.5.jar:/opt/storm/lib/logback-classic-1.0.13.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/conf:/opt/monitoring_agent/bobbrad.jar
When the supervisor starts up you will see bobbrad.jar in the classpath. I see the code in
supervisor.clj (defn resources-jar []
  (->> (.split (current-classpath) File/pathSeparator)
       (filter #(.endsWith  % ".jar"))
       (filter #(zip-contains-dir? % RESOURCES-SUBDIR))
       first ))It does not seem this code is the source of this running with zookeeper cluster.
 Suprivisor is applying what comes down from Zookeeper java.classpath.Can someone point me
to where that is happening in a cluster environment.?Thanks for any help. 
 		 	   		   		 	   		  
Mime
View raw message