qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kim van der Riet (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (QPIDIT-93) Optionally produce xUnit XML report with test results
Date Fri, 26 Jan 2018 15:00:00 GMT

    [ https://issues.apache.org/jira/browse/QPIDIT-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16341138#comment-16341138
] 

Kim van der Riet commented on QPIDIT-93:
----------------------------------------

With the above commit, I have taken the following approach:

As all the test information and results are contained within the {{unittest.TestCase}} and
{{unittest.TestResult}} objects, I created a new {{xunit_log}} module which takes these
and generates an xUnit-formatted report using the standard Python {{xml.etree.cElementTree}} module
to construct the XML.  To activate the xUnit logging, the {{--xunit-log}} parameter flag
must be present when running the test(s).

The xUnit XML is based on this specification found at [GitHub|https://gist.github.com/erikd/4192748] (attached),
a sample of the output is also attached.

The only information valid to a test report, but which does not appear in the xUnit XSD is
a description of the broker (and optionally its config) and a full list of parameter options
(for example {{--no-skip --broker-type None --xunit-log}}) used in the test. For example,
the broker could be one of several and which would produce different results. Additionally,
for some configurations (for example, using the [Qpid Dispatch Router|https://qpid.apache.org/components/dispatch-router/index.html]),
the configuration of nodes and brokers (if any) is also relevant.  It feels to me as though
there needs to be broker name, broker configuration and test parameters elements and/or attributes
in the schema somewhere. This particular schema does not seem to be extensible. Am I missing
something?

Ideas welcome.

> Optionally produce xUnit XML report with test results
> -----------------------------------------------------
>
>                 Key: QPIDIT-93
>                 URL: https://issues.apache.org/jira/browse/QPIDIT-93
>             Project: Apache QPID Interoperability Test Suite
>          Issue Type: Improvement
>    Affects Versions: 0.1.0
>            Reporter: Jiri Daněk
>            Assignee: Kim van der Riet
>            Priority: Minor
>         Attachments: amqp_types_test.2018-01-26T09-24-24.xml, xunit.xsd
>
>
> Something like the following should do the job. Instead of https://pypi.python.org/pypi/unittest-xml-reporting,
it might be possible to maybe switch from unittest package to py.test, which has xUnit reports
built in.
> {noformat}
> diff --git a/src/python/qpid_interop_test/jms_messages_test.py b/src/python/qpid_interop_test/jms_messages_test.py
> index 3b54510..3b94f3a 100755
> --- a/src/python/qpid_interop_test/jms_messages_test.py
> +++ b/src/python/qpid_interop_test/jms_messages_test.py
> @@ -24,19 +24,17 @@ Module to test JMS message types across different APIs
>  from json import dumps
>  from os import getenv, path
> +import xmlrunner
>  from proton import symbol
>  from qpid_interop_test.test_type_map import TestTypeMap
> @@ -302,7 +300,7 @@ class TestOptions(object):
>      Class controlling command-line arguments used to control the test.
>      """
>      def __init__(self, shim_map):
> -        parser = argparse.ArgumentParser(description='Qpid-interop AMQP client interoparability
test suite '
> +        parser = argparse.ArgumentParser(description='Qpid-interop AMQP client interoperability
test suite '
>                                           'for JMS message types')
>          parser.add_argument('--sender', action='store', default='localhost:5672', metavar='IP-ADDR:PORT',
>                              help='Node to which test suite will send messages.')
> @@ -313,6 +311,8 @@ class TestOptions(object):
>          parser.add_argument('--broker-type', action='store', metavar='BROKER_NAME',
>                              help='Disable test of broker type (using connection properties)
by specifying the broker' +
>                              ' name, or "None".')
> +        parser.add_argument('--xunit-report-dir', action='store', metavar='REPORTS_DIRECTORY',
> +                            help='Generate xUnit report into REPORTS_DIRECTORY/TEST-<test_name>-<timestamp>.xml')
>          type_group = parser.add_mutually_exclusive_group()
>          type_group.add_argument('--include-type', action='append', metavar='JMS_MESSAGE-TYPE',
>                                  help='Name of JMS message type to include. Supported
types:\n%s' %
> @@ -421,6 +421,7 @@ if __name__ == '__main__':
>              TEST_SUITE.addTest(unittest.makeSuite(test_case_class))
>  
>      # Finally, run all the dynamically created tests
> -    RES = unittest.TextTestRunner(verbosity=2).run(TEST_SUITE)
> +#    RES = unittest.TextTestRunner(verbosity=2).run(TEST_SUITE)  # type: unittest.TextTestResult
> +    RES = xmlrunner.XMLTestRunner(verbosity=2).run(TEST_SUITE)  # type: unittest.TextTestResult
>      if not RES.wasSuccessful():
>          sys.exit(1)
> {noformat}
> The main "problem" seems that each test has completely separate main method and builds
up options from scratch. Adding this to every test would mean some unnecessary duplication.
So maybe refactor this somewhat at first? On the other hand, there are good reasons to keep
the tests simple...
> What do you think?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message