metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-1810) Storm Profiler Intermittent Test Failure
Date Thu, 06 Dec 2018 18:50:00 GMT

    [ https://issues.apache.org/jira/browse/METRON-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16711858#comment-16711858
] 

ASF GitHub Bot commented on METRON-1810:
----------------------------------------

Github user justinleet commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1289#discussion_r239568177
  
    --- Diff: metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/DefaultMessageDistributor.java
---
    @@ -315,41 +308,51 @@ public DefaultMessageDistributor withPeriodDuration(int duration,
TimeUnit units
       }
     
       /**
    -   * A listener that is notified when profiles expire from the active cache.
    +   * Notified synchronously when the active cache is modified.
        */
    -  private class ActiveCacheRemovalListener implements RemovalListener<Integer, ProfileBuilder>,
Serializable {
    +  private class ActiveCacheWriter implements CacheWriter<Integer, ProfileBuilder>,
Serializable {
    +
    +    @Override
    +    public void write(@Nonnull Integer key, @Nonnull ProfileBuilder value) {
    +      // do nothing
    +    }
     
         @Override
    -    public void onRemoval(@Nullable Integer key, @Nullable ProfileBuilder expired, @Nonnull
RemovalCause cause) {
    -      LOG.warn("Profile expired from active cache; profile={}, entity={}",
    -              expired.getDefinition().getProfile(),
    -              expired.getEntity());
    +    public void delete(@Nonnull Integer key, @Nullable ProfileBuilder value, @Nonnull
RemovalCause cause) {
    +      if(cause.wasEvicted()) {
    +        // add the profile to the expired cache
    +        expiredCache.put(key, value);
    +        LOG.info("Profile expired from active cache due to inactivity; profile={}, entity={},
cause={}",
    --- End diff --
    
    Will this LOG.info cause a lot of churn in the logs? In a production scenario, there could
be a lot of entities per profile, right? And these could be expiring pretty often.  It seems
like you could easily hit scenarios where thousands of entities are expiring and then clogging
the logger up.  This seems like more a debugging log than anything else.


> Storm Profiler Intermittent Test Failure
> ----------------------------------------
>
>                 Key: METRON-1810
>                 URL: https://issues.apache.org/jira/browse/METRON-1810
>             Project: Metron
>          Issue Type: Bug
>            Reporter: Nick Allen
>            Priority: Major
>              Labels: test-failure
>
> {code:java}
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 289.466 sec <<<
FAILURE! - in org.apache.metron.profiler.storm.integration.ProfilerIntegrationTest
> testProcessingTimeWithTimeToLiveFlush(org.apache.metron.profiler.storm.integration.ProfilerIntegrationTest)
 Time elapsed: 136.722 sec  <<< FAILURE!
> java.lang.AssertionError
> 	at org.junit.Assert.fail(Assert.java:86)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.junit.Assert.assertTrue(Assert.java:52)
> 	at org.apache.metron.profiler.storm.integration.ProfilerIntegrationTest.testProcessingTimeWithTimeToLiveFlush(ProfilerIntegrationTest.java:210)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
> 	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> Results :
> Failed tests: 
>   ProfilerIntegrationTest.testProcessingTimeWithTimeToLiveFlush:210 null{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message