metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-1916) Stellar Classpath Function Resolver Should Handle ClassNotFoundException
Date Wed, 05 Dec 2018 23:22:00 GMT

    [ https://issues.apache.org/jira/browse/METRON-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710739#comment-16710739
] 

ASF GitHub Bot commented on METRON-1916:
----------------------------------------

Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/1288
  
    It was when I was working on #1289  .  After changing the POM, I found that multiple versions
of httpclient were getting pulled in, which broke all the integration tests because a Stellar
executor could not load.  It would throw the exception that I put in the JIRA.
    
    This is one specific scenario, but I can imagine that classpath issues like this are going
to crop-up in different environments.  It seems like we should have a way to handle this.


> Stellar Classpath Function Resolver Should Handle ClassNotFoundException
> ------------------------------------------------------------------------
>
>                 Key: METRON-1916
>                 URL: https://issues.apache.org/jira/browse/METRON-1916
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Nick Allen
>            Assignee: Nick Allen
>            Priority: Major
>
> Stellar can be executed in an environment where not all of the dependencies are available
for each of the functions provided in Stellar Common. When this happens, the classpath resolver
blows-up even if you do not use the function that is missing dependencies.
> For example this exception is thrown when not all of the dependencies are available for
the new REST function.  I was only attempting to use STATS_INIT, not the REST function.
> {code:java}
> 2018-11-30 15:47:00 DEBUG BaseFunctionResolver:165 - Resolving Stellar function in class
RestFunctions.RestGet
> 2018-11-30 15:47:00 ERROR DefaultProfileBuilder:302 - Bad 'init' expression: error='Unable
to parse: STATS_INIT() due to: org/apache/http/conn/HttpClientConnectionManager', expr='STATS_INIT()',
profile='profile-with-stats', entity='global', variables-available='[adapter.threatinteladapter.end.ts,
bro_timestamp, ip_dst_port, enrichmentsplitterbolt.splitter.end.ts, enrichmentsplitterbolt.splitter.begin.ts,
adapter.hostfromjsonlistadapter.end.ts, adapter.geoadapter.begin.ts, uid, trans_depth, protocol,
original_string, ip_dst_addr, threatinteljoinbolt.joiner.ts, host, enrichmentjoinbolt.joiner.ts,
adapter.hostfromjsonlistadapter.begin.ts, threatintelsplitterbolt.splitter.begin.ts, ip_src_addr,
user_agent, timestamp, method, request_body_len, uri, tags, source.type, adapter.geoadapter.end.ts,
referrer, threatintelsplitterbolt.splitter.end.ts, adapter.threatinteladapter.begin.ts, ip_src_port,
guid, response_body_len]'
> org.apache.metron.stellar.dsl.ParseException: Unable to parse: STATS_INIT() due to: org/apache/http/conn/HttpClientConnectionManager
> at org.apache.metron.stellar.common.BaseStellarProcessor.createException(BaseStellarProcessor.java:166)
> at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:154)
> at org.apache.metron.stellar.common.DefaultStellarStatefulExecutor.execute(DefaultStellarStatefulExecutor.java:160)
> at org.apache.metron.stellar.common.DefaultStellarStatefulExecutor.assign(DefaultStellarStatefulExecutor.java:95)
> at org.apache.metron.profiler.DefaultProfileBuilder.assign(DefaultProfileBuilder.java:291)
> at org.apache.metron.profiler.DefaultProfileBuilder.apply(DefaultProfileBuilder.java:136)
> at org.apache.metron.profiler.DefaultMessageDistributor.distribute(DefaultMessageDistributor.java:177)
> at org.apache.metron.profiler.storm.ProfileBuilderBolt.handleMessage(ProfileBuilderBolt.java:411)
> at org.apache.metron.profiler.storm.ProfileBuilderBolt.execute(ProfileBuilderBolt.java:337)
> at org.apache.storm.topology.WindowedBoltExecutor$1.onActivation(WindowedBoltExecutor.java:307)
> at org.apache.storm.windowing.WindowManager.onTrigger(WindowManager.java:145)
> at org.apache.storm.windowing.WatermarkTimeTriggerPolicy.handleWaterMarkEvent(WatermarkTimeTriggerPolicy.java:80)
> at org.apache.storm.windowing.WatermarkTimeTriggerPolicy.track(WatermarkTimeTriggerPolicy.java:48)
> at org.apache.storm.windowing.WindowManager.track(WindowManager.java:177)
> at org.apache.storm.windowing.WindowManager.add(WindowManager.java:110)
> at org.apache.storm.windowing.WaterMarkEventGenerator.run(WaterMarkEventGenerator.java:80)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoClassDefFoundError: org/apache/http/conn/HttpClientConnectionManager
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.createFunction(BaseFunctionResolver.java:240)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.resolveFunction(BaseFunctionResolver.java:194)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.resolveFunctions(BaseFunctionResolver.java:166)
> at com.google.common.base.Suppliers$MemoizingSupplier.get(Suppliers.java:109)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:149)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:49)
> at org.apache.metron.stellar.common.StellarCompiler.resolveFunction(StellarCompiler.java:688)
> at org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:656)
> at org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259)
> at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151)
> ... 21 more{code}
> When this happens the Classpath Function Resolver should allow me to use whichever functions
can be loaded correctly.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message