metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otto Fowler <ottobackwa...@gmail.com>
Subject Re: [DISCUSS] Storm worker processes not properly terminating on restart (METRON-485)
Date Mon, 24 Oct 2016 19:15:44 GMT
Can you try that from the command line ( leave monit out )?  I think monit
has it’s own command timeout and that will simplify.



On October 24, 2016 at 14:10:41, Zeolla@GMail.com (zeolla@gmail.com) wrote:

Sorry for the delay here guys - I've added the -w 60 param to
/usr/local/monit/stop_*_topology.sh and I'm still seeing threads get left
behind during reboots.

Jon

On Wed, Oct 5, 2016 at 3:54 PM Zeolla@GMail.com <zeolla@gmail.com> wrote:

> Right, I will try that.  I guess my main question, though, is why would
> that need more time?  Is it a function of CPU, memory, disk, or otherwise?
> Obviously if the timeout is the issue it's probably related to what I'm
> sending into the cluster and how resources are allocated in relation to the
> workload but I plan to scale this up roughly 10x and don't want to end up
> with a 600 second timeout.
>
> Jon
>
> On Wed, Oct 5, 2016, 15:34 Otto Fowler <ottobackwards@gmail.com> wrote:
>
> What you would have to do to try this is to edit
> /usr/local/monit/stop_bro_topology.sh and add in a -w parameter.
>  to read:
>
> storm kill bro -w 60
>
> save that and then try to stop if from monit again.
>
> If that works, then we will know.
>
> On October 5, 2016 at 15:18:43, Zeolla@GMail.com (zeolla@gmail.com) wrote:
>
> topology.message.timeout.secs is set to 30 according to ambari.
>
> What you requested is essentially what I did before, except instead of
> using `storm kill` I used monit to kill all of the topologies, and I waited
> multiple minutes before stopping nimbus/supervisor/drpc, then waited a
> couple more minutes before grabbing the prior metrics/processes. I've seen
> this multiple times across the past week and each time I see it I shut
> everything down, do a `kill -9` on the storm processes left behind, make
> sure no processes are running as the storm user on my entire cluster, then
> bring things up in ambari, then monit once ambari says
> nimbus/supervisor/drpc is up.
>
> Jon
>
> On Wed, Oct 5, 2016 at 3:09 PM P. Taylor Goetz <ptgoetz@gmail.com> wrote:
>
> > To build on what Otto said:
> >
> > What’s your topology.message.timeout.secs configuration value? The
> default
> > is 30.
> >
> > If you issue the `storm kill` command without the -w option, Storm will
> > first deactivate the topology, then wait topology.message.timeout.secs
> > before actually shutting down the topology.
> >
> > Also, if you have topologies deployed, and stop ninmubs, supervisors,
> etc.
> > the workers may continue to run (depending on how you have monit setup).
> >
> > Can you try the following:
> >
> > 1. Kill the topology and wait topology.message.timeout.sec so Storm
> > properly shuts down the topology.
> > 2. Stop nimbus/supervisor/etc.
> > 3. Check to see if you still have threads left behind.
> >
> >
> > -Taylor
> >
> > > On Oct 5, 2016, at 2:31 PM, Otto Fowler <ottobackwards@gmail.com>
> wrote:
> > >
> > > The shutdown topology scripts only do a storm kill <item>.
> > > They probably need to -w WAITTIME.
> > >
> > >
> > > --
> > >
> > > Sent with Airmail
> > >
> > > On October 5, 2016 at 13:53:33, Zeolla@GMail.com (zeolla@gmail.com)
> > wrote:
> > >
> > > Hi All,
> > >
> > > While tuning Metron I have been doing a large amount of storm topology
> > > restarts, as I tune the parallelism and memory use of bolts. I have
> > > noticed multiple times that some threads will be left behind on
> restart,
> > > which eventually causes all topologies to fail to start due to
> > > "java.lang.OutOfMemoryError: unable to create new native thread". I am
> > > only using the bro, indexing, and enrichments topologies right now,
> and I
> > > have seen all of them leave hanging processes behind. Has anybody else
> > > seen anything like this?
> > >
> > > I'm exclusively using monit to do topology restarts, and Ambari to
> > restart
> > > supervisors/nimbus/UI/DRPC processes. When it is currently registering
> as
> > > stopped in both monit and Ambari, I see the following threads left
> > behind:
> > >> [jzeolla@node1 storm]$ ps -austorm | wc -l
> > >> 6
> > >> [jzeolla@node2 storm]$ ps -austorm | wc -l
> > >> 12
> > >> [jzeolla@node3 storm]$ ps -austorm | wc -l
> > >> 9
> > >> [jzeolla@node4 storm]$ ps -austorm | wc -l
> > >> 11
> > >> [jzeolla@node5 storm]$ ps -austorm | wc -l
> > >> 9
> > >
> > > Some sample processes:
> > >> storm 14257 0.1 0.2 6279184 293912 ? Sl Oct03 3:44
> > > /usr/jdk64/jdk1.8.0_60/bin/java -server -Ddaemon.name=logviewer
> > > -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.0.0-169/storm
> > > -Dstorm.log.dir=/var/log/storm
> > >
> >
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dstorm.conf.file= -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/
> > >
> > > 2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.
> >
> 0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm:/usr/hdp/current/storm-supervisor/conf
> > >
> > > -Xmx128m -Dlogfile.name=logviewer.log
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/cluster.xml
> > > backtype.storm.daemon.logviewer
> > >> storm 32072 0.0 0.1 37337056 177112 ? Sl Oct02 2:02
> > > /usr/jdk64/jdk1.8.0_60/bin/java -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/
> > >
> > > 2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.
> >
> 0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > -Dlogfile.name=bro-13-1475456606-worker-6704.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.id=bro-13-1475456606
> > > -Dworker.id=f10b3f2f-3ef7-4e8c-b9d3-10e7fabdc4d7 -Dworker.port=6704
> > > -Dstorm.log.dir=/var/log/storm
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > backtype.storm.LogWriter /usr/jdk64/jdk1.8.0_60/bin/java -server
> -Xmx20G
> > >
> >
> -javaagent:/usr/hdp/current/storm-client/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=localhost,port=8650,wireformat31x=true,mode=multicast,config=/usr/hdp/current/storm-client/contrib/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_6704_JVM
> > >
> > >
> >
> -Djava.library.path=/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/resources/Linux-amd64:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/resources:/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dlogfile.name=bro-13-1475456606-worker-6704.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.conf.file=
> > -Dstorm.options=
> > > -Dstorm.log.dir=/var/log/storm -Dlogging.sensitivity=S3
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > -Dstorm.id=bro-13-1475456606
> > > -Dworker.id=f10b3f2f-3ef7-4e8c-b9d3-10e7fabdc4d7 -Dworker.port=6704 -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/
> > >
> > > 2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.
> >
> 0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > backtype.storm.daemon.worker bro-13-1475456606
> > > 05236488-ed9a-4969-a74f-cb471d0e8fdf 6704
> > > f10b3f2f-3ef7-4e8c-b9d3-10e7fabdc4d7
> > >> storm 32127 121 7.4 32422464 9799588 ? Sl Oct02 4337:42
> > > /usr/jdk64/jdk1.8.0_60/bin/java -server -Xmx20G
> > >
> >
> -javaagent:/usr/hdp/current/storm-client/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=localhost,port=8650,wireformat31x=true,mode=multicast,config=/usr/hdp/current/storm-client/contrib/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_6704_JVM
> > >
> > >
> >
> -Djava.library.path=/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/resources/Linux-amd64:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/resources:/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dlogfile.name=bro-13-1475456606-worker-6704.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.conf.file=
> > -Dstorm.options=
> > > -Dstorm.log.dir=/var/log/storm -Dlogging.sensitivity=S3
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > -Dstorm.id=bro-13-1475456606
> > > -Dworker.id=f10b3f2f-3ef7-4e8c-b9d3-10e7fabdc4d7 -Dworker.port=6704 -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/
> > >
> > > 2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.
> >
> 0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > backtype.storm.daemon.worker bro-13-1475456606
> > > 05236488-ed9a-4969-a74f-cb471d0e8fdf 6704
> > > f10b3f2f-3ef7-4e8c-b9d3-10e7fabdc4d7
> > >> storm 47882 0.0 0.1 37338556 188920 ? Sl Oct04 0:47
> > > /usr/jdk64/jdk1.8.0_60/bin/java -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/
> > >
> > > 2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.
> >
> 0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/indexing-16-1475589543/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > -Dlogfile.name=indexing-16-1475589543-worker-6700.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm
> -Dstorm.id=indexing-16-1475589543
> > > -Dworker.id=5e17c5c3-e6ab-4b48-a48a-dc540cda7087 -Dworker.port=6700
> > > -Dstorm.log.dir=/var/log/storm
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > backtype.storm.LogWriter /usr/jdk64/jdk1.8.0_60/bin/java -server -Xmx2G
> > >
> >
> -javaagent:/usr/hdp/current/storm-client/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=localhost,port=8650,wireformat31x=true,mode=multicast,config=/usr/hdp/current/storm-client/contrib/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_6700_JVM
> > >
> > >
> >
> -Djava.library.path=/Data_1/hadoop/storm/supervisor/stormdist/indexing-16-1475589543/resources/Linux-amd64:/Data_1/hadoop/storm/supervisor/stormdist/indexing-16-1475589543/resources:/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dlogfile.name=indexing-16-1475589543-worker-6700.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.conf.file=
> > -Dstorm.options=
> > > -Dstorm.log.dir=/var/log/storm -Dlogging.sensitivity=S3
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > -Dstorm.id=indexing-16-1475589543
> > > -Dworker.id=5e17c5c3-e6ab-4b48-a48a-dc540cda7087 -Dworker.port=6700 -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/
> > >
> > > 2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.
> >
> 0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/indexing-16-1475589543/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > backtype.storm.daemon.worker indexing-16-1475589543
> > > 05236488-ed9a-4969-a74f-cb471d0e8fdf 6700
> > > 5e17c5c3-e6ab-4b48-a48a-dc540cda7087
> > >> storm 47978 4.7 0.5 19996036 740372 ? Sl Oct04 65:04
> > > /usr/jdk64/jdk1.8.0_60/bin/java -server -Xmx2G
> > >
> >
> -javaagent:/usr/hdp/current/storm-client/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=localhost,port=8650,wireformat31x=true,mode=multicast,config=/usr/hdp/current/storm-client/contrib/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_6700_JVM
> > >
> > >
> >
> -Djava.library.path=/Data_1/hadoop/storm/supervisor/stormdist/indexing-16-1475589543/resources/Linux-amd64:/Data_1/hadoop/storm/supervisor/stormdist/indexing-16-1475589543/resources:/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dlogfile.name=indexing-16-1475589543-worker-6700.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.conf.file=
> > -Dstorm.options=
> > > -Dstorm.log.dir=/var/log/storm -Dlogging.sensitivity=S3
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > -Dstorm.id=indexing-16-1475589543
> > > -Dworker.id=5e17c5c3-e6ab-4b48-a48a-dc540cda7087 -Dworker.port=6700 -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/
> > >
> > > 2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.
> >
> 0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/indexing-16-1475589543/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > backtype.storm.daemon.worker indexing-16-1475589543
> > > 05236488-ed9a-4969-a74f-cb471d0e8fdf 6700
> > > 5e17c5c3-e6ab-4b48-a48a-dc540cda7087
> > >> storm 21301 0.1 0.2 6347824 298632 ? Sl Oct03 3:44
> > > /usr/jdk64/jdk1.8.0_60/bin/java -server -Ddaemon.name=logviewer
> > > -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.0.0-169/storm
> > > -Dstorm.log.dir=/var/log/storm
> > >
> >
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dstorm.conf.file= -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/
> > >
> > >
> >
> 2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm:/usr/hdp/current/storm-supervisor/conf
> > >
> > > -Xmx128m -Dlogfile.name=logviewer.log
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/cluster.xml
> > > backtype.storm.daemon.logviewer
> > >> storm 14046 102 19.3 42125944 25539856 ? Sl Oct03 2655:13
> > > /usr/jdk64/jdk1.8.0_60/bin/java -server -Xmx20G
> > >
> >
> -javaagent:/usr/hdp/current/storm-client/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=localhost,port=8650,wireformat31x=true,mode=multicast,config=/usr/hdp/current/storm-client/contrib/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_6700_JVM
> > >
> > >
> >
> -Djava.library.path=/Data_1/hadoop/storm/supervisor/stormdist/enrichment-12-1475456272/resources/Linux-amd64:/Data_1/hadoop/storm/supervisor/stormdist/enrichment-12-1475456272/resources:/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dlogfile.name=enrichment-12-1475456272-worker-6700.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.conf.file=
> > -Dstorm.options=
> > > -Dstorm.log.dir=/var/log/storm -Dlogging.sensitivity=S3
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > -Dstorm.id=enrichment-12-1475456272
> > > -Dworker.id=5d586063-e405-4a67-a644-53a0ac943ab0 -Dworker.port=6700 -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/
> > >
> > >
> >
> 2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/enrichment-12-1475456272/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > backtype.storm.daemon.worker enrichment-12-1475456272
> > > eff7f971-ebe2-4f53-beab-99b061bfde10 6700
> > > 5d586063-e405-4a67-a644-53a0ac943ab0
> > >> storm 6544 0.0 0.1 37339132 180872 ? Sl Oct02 3:22
> > > /usr/jdk64/jdk1.8.0_60/bin/java -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/
> > >
> > >
> >
> 2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > -Dlogfile.name=bro-13-1475456606-worker-6704.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.id=bro-13-1475456606
> > > -Dworker.id=f917291c-bd1e-4a9c-a52d-b9c5fef0567e -Dworker.port=6704
> > > -Dstorm.log.dir=/var/log/storm
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > backtype.storm.LogWriter /usr/jdk64/jdk1.8.0_60/bin/java -server
> -Xmx20G
> > >
> >
> -javaagent:/usr/hdp/current/storm-client/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=localhost,port=8650,wireformat31x=true,mode=multicast,config=/usr/hdp/current/storm-client/contrib/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_6704_JVM
> > >
> > >
> >
> -Djava.library.path=/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/resources/Linux-amd64:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/resources:/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dlogfile.name=bro-13-1475456606-worker-6704.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.conf.file=
> > -Dstorm.options=
> > > -Dstorm.log.dir=/var/log/storm -Dlogging.sensitivity=S3
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > -Dstorm.id=bro-13-1475456606
> > > -Dworker.id=f917291c-bd1e-4a9c-a52d-b9c5fef0567e -Dworker.port=6704 -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/
> > >
> > >
> >
> 2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > backtype.storm.daemon.worker bro-13-1475456606
> > > eff7f971-ebe2-4f53-beab-99b061bfde10 6704
> > > f917291c-bd1e-4a9c-a52d-b9c5fef0567e
> > >> storm 6592 192 9.8 34986568 12987704 ? Sl Oct02 6866:30
> > > /usr/jdk64/jdk1.8.0_60/bin/java -server -Xmx20G
> > >
> >
> -javaagent:/usr/hdp/current/storm-client/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=localhost,port=8650,wireformat31x=true,mode=multicast,config=/usr/hdp/current/storm-client/contrib/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_6704_JVM
> > >
> > >
> >
> -Djava.library.path=/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/resources/Linux-amd64:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/resources:/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dlogfile.name=bro-13-1475456606-worker-6704.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.conf.file=
> > -Dstorm.options=
> > > -Dstorm.log.dir=/var/log/storm -Dlogging.sensitivity=S3
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > -Dstorm.id=bro-13-1475456606
> > > -Dworker.id=f917291c-bd1e-4a9c-a52d-b9c5fef0567e -Dworker.port=6704 -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/
> > >
> > >
> >
> 2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/bro-13-1475456606/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > backtype.storm.daemon.worker bro-13-1475456606
> > > eff7f971-ebe2-4f53-beab-99b061bfde10 6704
> > > f917291c-bd1e-4a9c-a52d-b9c5fef0567e
> > >> storm 13998 0.0 0.1 37339104 184140 ? Sl Oct03 1:57
> > > /usr/jdk64/jdk1.8.0_60/bin/java -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/
> > >
> > >
> >
> 2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/enrichment-12-1475456272/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > -Dlogfile.name=enrichment-12-1475456272-worker-6700.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm
> > -Dstorm.id=enrichment-12-1475456272
> > > -Dworker.id=5d586063-e405-4a67-a644-53a0ac943ab0 -Dworker.port=6700
> > > -Dstorm.log.dir=/var/log/storm
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > backtype.storm.LogWriter /usr/jdk64/jdk1.8.0_60/bin/java -server
> -Xmx20G
> > >
> >
> -javaagent:/usr/hdp/current/storm-client/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=localhost,port=8650,wireformat31x=true,mode=multicast,config=/usr/hdp/current/storm-client/contrib/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_6700_JVM
> > >
> > >
> >
> -Djava.library.path=/Data_1/hadoop/storm/supervisor/stormdist/enrichment-12-1475456272/resources/Linux-amd64:/Data_1/hadoop/storm/supervisor/stormdist/enrichment-12-1475456272/resources:/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >
> > > -Dlogfile.name=enrichment-12-1475456272-worker-6700.log
> > > -Dstorm.home=/usr/hdp/2.4.0.0-169/storm -Dstorm.conf.file=
> > -Dstorm.options=
> > > -Dstorm.log.dir=/var/log/storm -Dlogging.sensitivity=S3
> > > -Dlog4j.configurationFile=/usr/hdp/2.4.0.0-169/storm/log4j2/worker.xml
> > >
> >
> -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector
> > >
> > > -Dstorm.id=enrichment-12-1475456272
> > > -Dworker.id=5d586063-e405-4a67-a644-53a0ac943ab0 -Dworker.port=6700 -cp
> > >
> >
> /usr/hdp/2.4.0.0-169/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.0.0-169/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-core-1.1.5.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/asm-4.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.0.0-169/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.0.0-169/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.0.0-169/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/storm-core-0.10.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/ring-json-0.3.1.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.0.0-169/storm/lib/commons-codec-1.6.jar:/usr/hdp/
> > >
> > > 2.4.0.
> > >
> >
> 0-169/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.0.0-169/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.0.0-169/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.0.0-169/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.0.0-169/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.0.0-169/storm/lib/zookeeper.jar:/usr/hdp/
> > >
> > >
> >
> 2.4.0.0-169/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-storm-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/storm/extlib-daemon/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/current/storm-supervisor/contrib/storm-jmxetric/lib/jmxetric-1.0.4.jar:/Data_1/hadoop/storm/supervisor/stormdist/enrichment-12-1475456272/stormjar.jar:/etc/hbase/conf:/etc/hadoop/conf
> > >
> > > backtype.storm.daemon.worker enrichment-12-1475456272
> > > eff7f971-ebe2-4f53-beab-99b061bfde10 6700
> > > 5d586063-e405-4a67-a644-53a0ac943ab0
> > >
> > >
> > > Logs showing the inability to start a topology:
> > >> 2016-10-04 10:10:57.787 b.s.m.n.Client [INFO] creating Netty Client,
> > > connecting to pgh-dat-t53.iso.local.cmu.edu:6702, bufferSize: 5242880
> > >> 2016-10-04 10:10:57.787 o.a.s.c.r.ExponentialBackoffRetry [WARN]
> > > maxRetries too large (30). Pinning to 29
> > >> 2016-10-04 10:10:57.787 b.s.u.StormBoundedExponentialBackoffRetry
> [INFO]
> > > The baseSleepTimeMs [100] the maxSleepTimeMs [1000] the maxRetries [30]
> > >> 2016-10-04 10:10:57.787 b.s.m.n.Client [INFO] connection attempt 1 to
> > > Netty-Client-pgh-dat-t53.iso.local.cmu.edu/172.19.247.79:6702
> scheduled
> > to
> > > run in 0 ms
> > >> 2016-10-04 10:10:57.790 b.s.m.n.Client [INFO] connection established
> to
> > > Netty-Client-pgh-dat-t53.iso.local.cmu.edu/172.19.247.79:6703
> > >> 2016-10-04 10:10:57.790 b.s.m.n.Client [INFO] connection established
> to
> > > Netty-Client-pgh-dat-t53.iso.local.cmu.edu/172.19.247.79:6702
> > >> 2016-10-04 10:10:57.790 b.s.m.n.Client [INFO] connection established
> to
> > > Netty-Client-pgh-dat-t53.iso.local.cmu.edu/172.19.247.79:6701
> > >> 2016-10-04 10:10:58.030 b.s.d.executor [INFO] Loading executor
> > > enrichmentJoinBolt:[2 2]
> > >> 2016-10-04 10:10:58.064 b.s.d.executor [INFO] Loaded executor tasks
> > > enrichmentJoinBolt:[2 2]
> > >> 2016-10-04 10:10:58.071 b.s.d.executor [INFO] Finished loading
> executor
> > > enrichmentJoinBolt:[2 2]
> > >> 2016-10-04 10:10:58.081 b.s.d.executor [INFO] Loading executor
> > > simpleHBaseThreatIntelBolt:[22 22]
> > >> 2016-10-04 10:10:58.095 b.s.d.executor [INFO] Loaded executor tasks
> > > simpleHBaseThreatIntelBolt:[22 22]
> > >> 2016-10-04 10:10:58.097 b.s.d.executor [INFO] Finished loading
> executor
> > > simpleHBaseThreatIntelBolt:[22 22]
> > >> 2016-10-04 10:10:58.104 b.s.d.executor [INFO] Loading executor
> > > kafkaSpout:[18 18]
> > >> 2016-10-04 10:10:58.112 b.s.d.executor [INFO] Loaded executor tasks
> > > kafkaSpout:[18 18]
> > >> 2016-10-04 10:10:58.118 b.s.d.executor [INFO] Finished loading
> executor
> > > kafkaSpout:[18 18]
> > >> 2016-10-04 10:10:58.125 b.s.d.executor [INFO] Loading executor
> > > geoEnrichmentBolt:[6 6]
> > >> 2016-10-04 10:10:58.135 b.s.d.executor [INFO] Loaded executor tasks
> > > geoEnrichmentBolt:[6 6]
> > >> 2016-10-04 10:10:58.137 b.s.d.executor [ERROR]
> > >> java.lang.OutOfMemoryError: unable to create new native thread
> > >> at java.lang.Thread.start0(Native Method) ~[?:1.8.0_60]
> > >> at java.lang.Thread.start(Thread.java:714) ~[?:1.8.0_60]
> > >> at backtype.storm.util$async_loop.doInvoke(util.clj:491)
> > > ~[storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.RestFn.invoke(RestFn.java:559)
> > > ~[clojure-1.6.0.jar:?]
> > >> at backtype.storm.daemon.executor$fn__7245.invoke(executor.clj:820)
> > > ~[storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.MultiFn.invoke(MultiFn.java:236)
> > > ~[clojure-1.6.0.jar:?]
> > >> at
> > >
> >
> backtype.storm.daemon.executor$mk_executor$fn__7116.invoke(executor.clj:351)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at
> > > backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:350)
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at
> > >
> >
> backtype.storm.daemon.worker$fn__7719$exec_fn__1353__auto__$reify__7721$iter__7726__7730$fn__7731.invoke(worker.clj:541)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.6.0.jar:?]
> > >> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.6.0.jar:?]
> > >> at clojure.lang.Cons.next(Cons.java:39) [clojure-1.6.0.jar:?]
> > >> at clojure.lang.RT.next(RT.java:598) [clojure-1.6.0.jar:?]
> > >> at clojure.core$next.invoke(core.clj:64) [clojure-1.6.0.jar:?]
> > >> at clojure.core$dorun.invoke(core.clj:2856) [clojure-1.6.0.jar:?]
> > >> at clojure.core$doall.invoke(core.clj:2871) [clojure-1.6.0.jar:?]
> > >> at
> > >
> >
> backtype.storm.daemon.worker$fn__7719$exec_fn__1353__auto__$reify__7721.run(worker.clj:541)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at java.security.AccessController.doPrivileged(Native Method)
> > > ~[?:1.8.0_60]
> > >> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_60]
> > >> at
> > >
> >
> backtype.storm.daemon.worker$fn__7719$exec_fn__1353__auto____7720.invoke(worker.clj:515)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.AFn.applyToHelper(AFn.java:178)
> > > [clojure-1.6.0.jar:?]
> > >> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.6.0.jar:?]
> > >> at clojure.core$apply.invoke(core.clj:624) [clojure-1.6.0.jar:?]
> > >> at
> > >
> >
> backtype.storm.daemon.worker$fn__7719$mk_worker__7805.doInvoke(worker.clj:491)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.6.0.jar:?]
> > >> at backtype.storm.daemon.worker$_main.invoke(worker.clj:641)
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.AFn.applyToHelper(AFn.java:165)
> > > [clojure-1.6.0.jar:?]
> > >> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.6.0.jar:?]
> > >> at backtype.storm.daemon.worker.main(Unknown Source)
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> 2016-10-04 10:10:58.169 b.s.util [ERROR] Halting process: ("Worker
> > died")
> > >> java.lang.RuntimeException: ("Worker died")
> > >> at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:332)
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.6.0.jar:?]
> > >> at
> > > backtype.storm.daemon.worker$fn__7818$fn__7819.invoke(worker.clj:636)
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at
> > >
> >
> backtype.storm.daemon.executor$mk_executor_data$fn__7078$fn__7079.invoke(executor.clj:256)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at
> > >
> >
> backtype.storm.daemon.executor$mk_executor$fn__7116.invoke(executor.clj:350)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at
> > > backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:350)
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at
> > >
> >
> backtype.storm.daemon.worker$fn__7719$exec_fn__1353__auto__$reify__7721$iter__7726__7730$fn__7731.invoke(worker.clj:541)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.6.0.jar:?]
> > >> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.6.0.jar:?]
> > >> at clojure.lang.Cons.next(Cons.java:39) [clojure-1.6.0.jar:?]
> > >> at clojure.lang.RT.next(RT.java:598) [clojure-1.6.0.jar:?]
> > >> at clojure.core$next.invoke(core.clj:64) [clojure-1.6.0.jar:?]
> > >> at clojure.core$dorun.invoke(core.clj:2856) [clojure-1.6.0.jar:?]
> > >> at clojure.core$doall.invoke(core.clj:2871) [clojure-1.6.0.jar:?]
> > >> at
> > >
> >
> backtype.storm.daemon.worker$fn__7719$exec_fn__1353__auto__$reify__7721.run(worker.clj:541)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at java.security.AccessController.doPrivileged(Native Method)
> > > ~[?:1.8.0_60]
> > >> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_60]
> > >> at
> > >
> >
> backtype.storm.daemon.worker$fn__7719$exec_fn__1353__auto____7720.invoke(worker.clj:515)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.AFn.applyToHelper(AFn.java:178)
> > > [clojure-1.6.0.jar:?]
> > >> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.6.0.jar:?]
> > >> at clojure.core$apply.invoke(core.clj:624) [clojure-1.6.0.jar:?]
> > >> at
> > >
> >
> backtype.storm.daemon.worker$fn__7719$mk_worker__7805.doInvoke(worker.clj:491)
> > >
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.6.0.jar:?]
> > >> at backtype.storm.daemon.worker$_main.invoke(worker.clj:641)
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > >> at clojure.lang.AFn.applyToHelper(AFn.java:165)
> > > [clojure-1.6.0.jar:?]
> > >> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.6.0.jar:?]
> > >> at backtype.storm.daemon.worker.main(Unknown Source)
> > > [storm-core-0.10.0.2.4.0.0-169.jar:0.10.0.2.4.0.0-169]
> > > --
> > >
> > > Jon
> >
> > --
>
> Jon
>
> --
>
> Jon
>
--

Jon

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message