tinkerpop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stephen mallette (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (TINKERPOP-2013) Process tests that are auto-ignored stink
Date Thu, 13 Dec 2018 16:13:00 GMT

     [ https://issues.apache.org/jira/browse/TINKERPOP-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stephen mallette closed TINKERPOP-2013.
       Resolution: Done
    Fix Version/s: 3.3.5

Basically made it so that {{OptOut}} are explicit - no more tricky background auto-ignores
for translators/remotes.


> Process tests that are auto-ignored stink
> -----------------------------------------
>                 Key: TINKERPOP-2013
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-2013
>             Project: TinkerPop
>          Issue Type: Improvement
>          Components: test-suite
>    Affects Versions: 3.2.9
>            Reporter: stephen mallette
>            Assignee: stephen mallette
>            Priority: Major
>             Fix For: 3.4.0, 3.3.5
> I think I've had it with {{GremlinProcessRunner}} and any other related infrastructure
that auto-ignores batches of tests unknowingly. The typical scenario is that a new test gets
added, we build the project and everything works without failures. Then something starts misbehaving
and we wonder why. After a lot of cycles and looking in the wrong places we realize that the
tests that should have caught the problem are simply being ignored. 
> There was a reason for making things work this way as we had assumed that certain tests
failing in a certain way was expected behavior, but that is not always the case. We’ve learned
that assumption doesn’t always apply and it would be better to be more explicit in dealing
with those failures.
> Not sure how this is going to be resolved, but I think that something needs to change.
Perhaps we need to identify all the auto-ignored tests and do any of the following:
> 1. Make them work
> 2. Explicitly ignore them
> 3. Re-organize tests so that it's not assumed that they just are part of the process
> 4. Something else??
> May also need to extend this into all the ignored GLV tests that aren't supported for
various reasons. Perhaps the assertion language needs more expressiveness.

This message was sent by Atlassian JIRA

View raw message