metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-1916) Stellar Classpath Function Resolver Should Handle ClassNotFoundException
Date Sat, 01 Dec 2018 14:18:00 GMT

    [ https://issues.apache.org/jira/browse/METRON-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705840#comment-16705840
] 

ASF GitHub Bot commented on METRON-1916:
----------------------------------------

Github user ottobackwards commented on the issue:

    https://github.com/apache/metron/pull/1288
  
    So, my main concern here is the state of stellar _after_ loading in this condition.  What
happens if you call a function that failed to load?  What happens if it is nested in a match
or a lambda?
    
    Right now, it blows up, but when loaded everything is known to be 'ok'.  So this introduces
a new state where we are loaded, but things that were supposed to load ( stellar functions
introduced to the class path ) have failed.
    
    I'm not sure this is that simple.  Would we _ever_ for example accept this state in the
topology?


> Stellar Classpath Function Resolver Should Handle ClassNotFoundException
> ------------------------------------------------------------------------
>
>                 Key: METRON-1916
>                 URL: https://issues.apache.org/jira/browse/METRON-1916
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Nick Allen
>            Assignee: Nick Allen
>            Priority: Major
>
> Stellar can be executed in an environment where not all of the dependencies are available
for each of the functions provided in Stellar Common. When this happens, the classpath resolver
blows-up even if you do not use the function that is missing dependencies.
> For example this exception is thrown when not all of the dependencies are available for
the new REST function.  I was only attempting to use STATS_INIT, not the REST function.
> {code:java}
> 2018-11-30 15:47:00 DEBUG BaseFunctionResolver:165 - Resolving Stellar function in class
RestFunctions.RestGet
> 2018-11-30 15:47:00 ERROR DefaultProfileBuilder:302 - Bad 'init' expression: error='Unable
to parse: STATS_INIT() due to: org/apache/http/conn/HttpClientConnectionManager', expr='STATS_INIT()',
profile='profile-with-stats', entity='global', variables-available='[adapter.threatinteladapter.end.ts,
bro_timestamp, ip_dst_port, enrichmentsplitterbolt.splitter.end.ts, enrichmentsplitterbolt.splitter.begin.ts,
adapter.hostfromjsonlistadapter.end.ts, adapter.geoadapter.begin.ts, uid, trans_depth, protocol,
original_string, ip_dst_addr, threatinteljoinbolt.joiner.ts, host, enrichmentjoinbolt.joiner.ts,
adapter.hostfromjsonlistadapter.begin.ts, threatintelsplitterbolt.splitter.begin.ts, ip_src_addr,
user_agent, timestamp, method, request_body_len, uri, tags, source.type, adapter.geoadapter.end.ts,
referrer, threatintelsplitterbolt.splitter.end.ts, adapter.threatinteladapter.begin.ts, ip_src_port,
guid, response_body_len]'
> org.apache.metron.stellar.dsl.ParseException: Unable to parse: STATS_INIT() due to: org/apache/http/conn/HttpClientConnectionManager
> at org.apache.metron.stellar.common.BaseStellarProcessor.createException(BaseStellarProcessor.java:166)
> at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:154)
> at org.apache.metron.stellar.common.DefaultStellarStatefulExecutor.execute(DefaultStellarStatefulExecutor.java:160)
> at org.apache.metron.stellar.common.DefaultStellarStatefulExecutor.assign(DefaultStellarStatefulExecutor.java:95)
> at org.apache.metron.profiler.DefaultProfileBuilder.assign(DefaultProfileBuilder.java:291)
> at org.apache.metron.profiler.DefaultProfileBuilder.apply(DefaultProfileBuilder.java:136)
> at org.apache.metron.profiler.DefaultMessageDistributor.distribute(DefaultMessageDistributor.java:177)
> at org.apache.metron.profiler.storm.ProfileBuilderBolt.handleMessage(ProfileBuilderBolt.java:411)
> at org.apache.metron.profiler.storm.ProfileBuilderBolt.execute(ProfileBuilderBolt.java:337)
> at org.apache.storm.topology.WindowedBoltExecutor$1.onActivation(WindowedBoltExecutor.java:307)
> at org.apache.storm.windowing.WindowManager.onTrigger(WindowManager.java:145)
> at org.apache.storm.windowing.WatermarkTimeTriggerPolicy.handleWaterMarkEvent(WatermarkTimeTriggerPolicy.java:80)
> at org.apache.storm.windowing.WatermarkTimeTriggerPolicy.track(WatermarkTimeTriggerPolicy.java:48)
> at org.apache.storm.windowing.WindowManager.track(WindowManager.java:177)
> at org.apache.storm.windowing.WindowManager.add(WindowManager.java:110)
> at org.apache.storm.windowing.WaterMarkEventGenerator.run(WaterMarkEventGenerator.java:80)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoClassDefFoundError: org/apache/http/conn/HttpClientConnectionManager
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.createFunction(BaseFunctionResolver.java:240)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.resolveFunction(BaseFunctionResolver.java:194)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.resolveFunctions(BaseFunctionResolver.java:166)
> at com.google.common.base.Suppliers$MemoizingSupplier.get(Suppliers.java:109)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:149)
> at org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:49)
> at org.apache.metron.stellar.common.StellarCompiler.resolveFunction(StellarCompiler.java:688)
> at org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:656)
> at org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259)
> at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151)
> ... 21 more{code}
> When this happens the Classpath Function Resolver should allow me to use whichever functions
can be loaded correctly.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message