Test Descriptors

Each PySys test has a file named pysystest.* which contains the test descriptor information such as the title, whether it is skipped, modes it runs in, etc. This information is all stored in a TestDescriptor instance, which can be accessed within each test using self.descriptor.

It is possible to configure the descriptor values in an XML file (pysystest.xml) separate to your Python (e.g. run.py) but it is recommended in modern PySys projects to put the descriptor information and your test code into a single file, typically called pysystest.py.

Sample pysystest.py

Each descriptor value is populated with a __pysys_KEY__ = VALUE line.

# Sample pysystest.py file defining a test (part of the "cookbook" sample).

# Test titles should be concise but give a clear idea of what is in scope for this testcase.
#
# Good titles make it easy to find the test you need even when you have 100s/1000s of tests.
# Tests can be sorted by title so try to use common prefixes (e.g. ``Category - Title``) to group related tests
# together, and run ``pysys print -s=title`` to see how your title looks alongside the existing titles.
#
# Titles need to be human readable at-a-glance, so don't put ids (e.g. bug tracking numbers) in the title;
# the "purpose" or "traceability_ids" are a better place for those details.
#
__pysys_title__   = r""" My foobar tool - Argument parsing success and error cases """
#                        ===============================================================================
# The underlined line length guide comment (above) discourages excessively long titles.

# The purpose is a good place for a fuller description of what is in and out of scope for this particular testcase.
__pysys_purpose__ = r""" The purpose of this test is to check that
      argument parsing addresses these criteria:
              - Correctness
              - Clear error messages
      """

# It's useful to keep track of who created each test and/or made subsequent changes so you know who can help if you
# need help debugging a problem.
__pysys_authors__ = "userid1, userid2, Joe Bloggs"

# The date the test was created using pysys make. If you copy an existing test you'd have to manually set this.
__pysys_created__ = "2021-07-25"

# Use this to list traceability ids for the requirements validated by this test, such as defect or user story ids.
__pysys_traceability_ids__ = "Bug-1234, UserStory-456, UserRequirement_1a, UserRequirement_2c, Performance"

# Comment/uncomment this to mark this test as skipped, which will stop it from executing.
__pysys_skipped_reason__   = "Skipped until Bug-1234 is fixed"

# Specify the groups that this test will be tagged with, allowing them to be selected for inclusion/exclusion in test
# runs. Groups are usually named in camelCase. These groups are separated by commas, followed by a semi-colon and
# inherit=true/false which specifies whether groups from parent pysysdirconfigs are inherited by this test.
#
# The disableCoverage group is a special group used by code coverage writers to ensure coverage tools are disabled for
# tests that are performance-critical. By default groups inherit, but you can override that with "; inherit=false".
__pysys_groups__           = "performance, disableCoverage; inherit=true"

# Specify the list of modes this test can be run in.
#
# Like test ids, mode names are usually TitleCase, with multiple dimensions delimited by an ``_`` underscore,
# e.g. ``CompressionGZip_AuthNone``. Each mode has an associated parameter dictionary, available to tests
# as ``self.mode.params``.
#
# Test modes are configured with a Python lambda that returns a list of modes. The ``helper`` parameter is an instance
# of ``pysys.config.descriptor.TestModesConfigHelper`` providing access to the inherited modes (and other
# useful functions/fields). If the mode name is not explicitly provided, a default mode name is
# generated by concatenating the parameter values with ``_`` (with a ``paramName=`` prefix for any numeric/boolean
# values).
#
# In your Python lambda you can return a simple list of modes, or combine your own modes with inherited modes
# defined by parent pysysdirconfigs. You can also use the power of Python list comprehensions to exclude certain modes,
# perhaps based on dynamic information such as the operating system. Project properties can be accessed
# using ``helper.project.PROPERTY_NAME``. Avoid expensive operations such as reading the file system from your lambda
# if possible.
#
# An alternative to providing a list of dicts for each mode is to provide a dict whose keys are the mode names and
# values are dicts containing the parameters.
#
# By default the first mode in each dimension is designated a *primary* mode (one that executes by default even
# when no ``--modes`` argument is specified), but this can be overridden by setting ``'isPrimary': True/False``
# in the dict for any mode. When mode dimensions are combined, a mode is primary if all the modes it is derived from
# were designated primary. When using modes for different execution environments/browsers etc you probably want only
# the first (typically fastest/simplest/most informative) mode to be primary, on the other hand if using modes to
# execute the same Python logic against various input files/args you should usually set all of the modes to be primary.
#
# It's often useful to combine multiple mode 'dimensions', for example all the combinations of a list of web browsers
# with a list of databases, or compression methods and authentication types. Rather than writing out every combination
# manually, you can use the function ``helper.createModeCombinations`` to automatically generate all combinations.
#
__pysys_modes__ = lambda helper: [
              mode for mode in
                      helper.createModeCombinations( # Takes any number of mode lists as arguments and returns a single combined mode list

                              helper.inheritedModes,

                              {
                                              'CompressionNone': {'compressionType':None, 'isPrimary':True},
                                              'CompressionGZip': {'compressionType':'gzip'},
                              },

                              [
                                      {'auth':None}, # Mode name is optional
                                      {'auth':'OS'}, # In practice auth=OS modes will always be excluded since MyFunkyOS is a fictional OS
                              ],

                      )

              # This is Python list comprehension syntax for filtering the items in the list
              if (mode['auth'] != 'OS' or helper.import_module('sys').platform == 'MyFunkyOS')
      ]

# A parameterized test is one where the same Python logic should be executed with various different parameters. Each
# set of parameters results in its own mode, with the parameter dict available to the test as ``self.mode.params``.
# Behind the scenes, setting parameterized_test_modes is equivalent to combining inherited and/or explicit modes
# with the parameterized modes (using ``createModeCombinations``) and designating all the parameterized modes as
# primary (i.e. executed by default when no ``--modes`` argument is specified).
__pysys_parameterized_test_modes__ = {
              'Usage':        {'cmd': ['--help'], 'expectedExitStatus':'==0'},
              'BadPort':      {'cmd': ['--port', '-1'],  'expectedExitStatus':'!=0'},
              'MissingPort':  {'cmd': [],  'expectedExitStatus':'!=0'},
      }

# Specify as a floating point number an indicator of when to run the tests under this directory, relative to other
# tests/directories with a higher or lower hint.
# The default priority is 0.0 so set the hint to a higher value to execute tests later, or a negative value to execute
# tests earlier.
# Comment this out to inherit from parent pysysdirconfig.xml files.
__pysys_execution_order_hint__ = +100.0

# By default the test class uses this pysystest.py module, but it is possible to use a different path for the test
# (even an absolute path). If you want to use a single Python class for lots of tests, use a class with at least one
# Python package in it (or set the module to the special string "PYTHONPATH"), and make sure it's available on the
# project's <pythonpath>.
__pysys_python_class__     = "PySysTest"
__pysys_python_module__    = "${testRootDir}/pysys-extensions/MySharedTestClass.py" # or "PYTHONPATH"

# You can customize the Input/Output/Reference directory names if you wish (or even provide an absolute
# paths if needed). These can also be specified using the older names <output/input/reference> with a path= attribute.
# In practice it is usually best to set this configuration in pysysproject.xml or pysysdirconfig.xml rather than
# in individual tests.
__pysys_input_dir__        = "${testRootDir}/pysys-extensions/my_shared_input_files"
__pysys_reference_dir__    = "MyReference"
__pysys_output_dir__       = "MyOutput"

# The ability to add user-defined data to the test descriptor is mostly useful when using a shared Python class for
# lots of tests, or for passing data from a pysystest.* file in a language other than Python into the descriptor
# for reading by Python code.
__pysys_user_data__        = {

              'myTestDescriptorData': 'foobar',

        # If a static field of the same name exists on the test class, it will be overridden with the associated user-data
        # value, with some basic type coersion to match the default value of the static field where possible.

              # For long values such as paths if the value is to be converted to a list, newline and/or comma can be used as
              # delimiters and will be automatically converted.

              # You can use project property syntax (e.g. ${propName} or ${eval: xxx}) and these will be expanded
              # automatically.
              'myTestDescriptorPath': '''
                      foo/foo-${os_myThirdPartyLibraryVersion}
                      foo/bar, foo/baz

                      foo/bosh
              '''
      }
# You can also specify items for the user data dictionary with separate items as follows. For non-Python pysystest.* files
# you can optionally use dot syntax instead of underscore e.g. "__pysys_user_data.myOtherUserData__".
__pysys_user_data_myOtherUserData__ = "foo/foo-${os_myThirdPartyLibraryVersion}, foo/bar, foo/baz"

# It is also possible to provide the descriptor values using XML embedded in this file as follows. Note that parsing
# XML is relatively slow, so add this value only if you have a good reason.
__pysys_xml_descriptor__ = r"""
      <?xml version="1.0" encoding="utf-8"?>
      <pysystest>

      </pysystest>
"""

# All __pysys_XXX__ descriptor values must be specified before the import statements and class definition.

import pysys
from pysys.constants import *
from pysys.basetest import BaseTest

class PySysTest(BaseTest):
      def execute(self):
              pass

      def validate(self):
              pass

This is intended as a reference, and to provide somewhere to copy snippets from as needed. Do not copy the whole thing into your own tests as this example contains many fields that are only used for advanced cases. Instead use pysys make to create new tests.

For more information about the modes helper shown above, see pysys.config.descriptor.TestModesConfigHelper.

All descriptor values should go at the start of the file, before any import statements - this is important for efficient parsing. For optimum parsing performance, make sure your first import is an import XXX rather than a from XXX import YYY statement.

It is mandatory to provide a __pysys_title__ value, but everything else is optional. For ease of readability and performance it is best to use # to comment out any items where you aren’t setting a (non-default) value.

Non-Python files can include descriptor values using the same syntax, but since they are parsed with regular expressions rather than Python, all values (except numbers/booleans) must be wrapped in a string, ideally a r""" ... """ string. In non-Python files these can appear at any position in the file, for example inside a class or a comment. For example pysystest.cs for C#/ECMAScript (either embedded within a C# comment or as fields of your C# class).

Sample pysystest.xml

The following sample illustrates all the possible configuration options for a test’s pysystest file when using standalone XML (instead of pysystest.py), i.e. pysystest.xml. This is just different (XML) syntax for the same capabilities described above for pysystest.py.

<?xml version="1.0" encoding="utf-8"?>
<pysystest
      authors="userid1, userid2, Joe Bloggs" created="1999-12-31"
>

      <!-- Test titles should be concise but give a clear idea of what is in scope for this testcase.

      Good titles make it easy to find the test you need even when you have 100s/1000s of tests.
      Tests can be sorted by title so try to use common prefixes (e.g. ``Category: Title``) to group related tests
      together, and run ``pysys print -s=title`` to see how your title looks alongside the existing titles.

      Titles need to be human readable at-a-glance, so don't put ids (e.g. bug tracking numbers) in the title;
      the "purpose" or "requirements" are a better place for those details.

      The title can also be specified without XML as: __pysystest_title__ = """ ... """
      -->
      <title>My foobar tool - Argument parsing success and error cases</title>

      <!-- The purpose can also be specified without XML as: __pysystest_purpose__ = """ ... """
      -->
      <purpose><![CDATA[
              This is a good place for a fuller description of what is in and
              out of scope for this particular testcase.

      ]]></purpose>

      <!-- Comment/uncomment this to mark this test as skipped, which will stop it from executing. -->
      <skipped reason="This test is skipped until Bug-1234 is fixed"/>

      <!-- Specify the groups that this test will be tagged with, allowing them to be selected
      for inclusion/exclusion in test runs. Groups are usually named in camelCase.
      These groups are in addition to any from parent pysysdirconfigs if inherit=true.

      The disableCoverage group is a special group used by code coverage writers to ensure coverage tools are disabled for
      tests that are performance-critical.
      -->
      <groups inherit="true" groups="performance, disableCoverage"/>

      <!-- Specify the list of modes this test can be run in.

      Like test ids, mode names are usually TitleCase, with multiple dimensions delimited by an ``_`` underscore,
      e.g. ``CompressionGZip_AuthNone``.

      Test modes are configured with a Python lambda that returns a list of modes, using a single parameter which is
      an instance of ``pysys.config.descriptor.TestModesConfigHelper`` providing access to the inherited modes (and other
      useful functions/fields). Each mode in the returned list is defined by a dictionary containing parameters to be set
      on the test object and/or a ``mode`` name. If the mode name is not explicitly provided, a default mode name is
      generated by concatenating the parameter values with ``_`` (with a ``paramName=`` prefix for any numeric/boolean
      values).

      In your Python lambda you can return a simple list of modes, or combine your own modes with inherited modes
      defined by parent pysysdirconfigs. You can also use the power of Python list comprehensions to exclude certain modes,
      perhaps based on dynamic information such as the operating system. Project properties can be accessed
      using ``helper.project.PROPERTY_NAME``. Avoid expensive operations such as reading the file system from your lambda
      if possible.

      The first listed mode is the "primary" mode, which is the one that is used by default when executing tests with
      no ``-m`` argument.

      It's often useful to combine multiple mode 'dimensions', for example all the combinations of a list of web browsers
      with a list of databases, or compression methods and authentication types. Rather than writing out every combination
      manually, you can use the function ``helper.createModeCombinations`` to automatically generate all combinations.

      Modes can be used to define similar subtests that share the same test class logic, for example testing your
      application's output when given various different input test vectors. For this common use case, if you already (or
      plan to) define multiple execution modes inherited in a parent directory, you usually want to
      use ``helper.createModeCombinations(helper.inheritedModes, [...])`` in your test so that each of the subtests you
      define in that second argument are executed in each of the inherited modes (if any).

      A test can use self.mode to find out which mode it is executing and/or self.mode.params to access any parameters.

      The specified Python code is invoked using `pysys.utils.safeeval.safeEval`.

      -->
      <modes>
              lambda helper: [
                              mode for mode in
                                      helper.createModeCombinations( # Takes any number of mode lists as arguments and returns a single combined mode list
                                              helper.inheritedModes,
                                              {
                                                              'CompressionNone': {'compressionType':None, 'isPrimary':True},
                                                              'CompressionGZip': {'compressionType':'gzip'},
                                              },
                                              [
                                                      {'auth':None}, # Mode name is optional
                                                      {'auth':'OS'}, # In practice auth=OS modes will always be excluded since MyFunkyOS is a fictional OS
                                              ],

                                              # By default only the first mode in each list is "primary", so the test will only run in that one mode by
                                              # default during local development (unless you supply a ``--modes`` or ``--ci`` argument). This is optimal when
                                              # using modes to validate the same behaviour/conditions in different execution environments e.g.
                                              # browsers/databases etc. However when using modes to validate different *behaviours/conditions* (e.g. testing
                                              # out different command line options) using a single PySysTest class, then you should have all your modes as
                                              # "primary" as you want all of them to execute by default in a quick local test run.
                                              helper.makeAllPrimary(
                                                      {
                                                              'Usage':        {'cmd': ['--help'], 'expectedExitStatus':'==0'},
                                                              'BadPort':      {'cmd': ['--port', '-1'],  'expectedExitStatus':'!=0'},
                                                              'MissingPort':  {'cmd': [],  'expectedExitStatus':'!=0'},
                                                      }),
                                              )
                              # This is Python list comprehension syntax for filtering the items in the list
                              if (mode['auth'] != 'OS' or helper.import_module('sys').platform == 'MyFunkyOS')
                      ]
      </modes>

      <!--
      NB: Older (pre-2.0) PySys projects used a more basic and limited syntax for specifying modes:

              <modes inherit="true">
                      <mode>CompressionNone</mode>
                      <mode>CompressionGZip</mode>
              </modes>
      -->

      <!-- Specify as a floating point number an indicator of when to run the tests under
      this directory, relative to other tests/directories with a higher or lower hint.
      Empty string hint="" means inherit. The default priority is 0.0 so set the hint to a higher
      value to execute tests later, or a negative value to execute tests earlier.
      -->
      <execution-order hint="+100.0"/>

      <!-- By convention the test class uses module="run.py" located in the test directory, but
      it is possible to use a different path (even an absolute path). If you want to use
      a single Python class for lots of tests, omit the module= attribute, ensure the class has at least one level of
      Python package, and make sure it's available on the project's <pythonpath> .

      -->
      <class name="PySysTest" module="${testRootDir}/pysys-extensions/MySharedTestClass.py"/>

      <!-- You can customize the Input/Output/Reference directory names if you wish (or even provide an absolute
      paths if needed). These can also be specified using the older names <output/input/reference> with a path= attribute.
      -->
      <input-dir>${testRootDir}/pysys-extensions/my_shared_input_files</input-dir>
      <reference-dir>MyReference</reference-dir>
      <output-dir>MyOutput</output-dir>

      <data>
              <!-- The ability to add user-defined data to the test descriptor is mostly useful when using a
              shared Python class for lots of tests.

              Project properties (but not other user-data values) can be substituted into the value using ${...},
              and ${eval: xxx} syntax can be used to evaluate some Python code (with project properties as Python variables).

              If a static field of the same name exists on the test class, it will be overridden with the associated user-data
              value, with some basic type coersion to match the default value of the static field where possible.
              -->
              <user-data name="myTestDescriptorData" value="foobar"/>

              <!-- For long values such as paths the value can be specified in a text (or CDATA) node, and if the
              value is to be converted to a list, newline and/or comma can be used as delimiters. -->
              <user-data name="myTestDescriptorPath">
                      foo/foo-${os_myThirdPartyLibraryVersion}
                      foo/bar, foo/baz
                      <!-- Comments and whitespace are ignored when converting a string to a list. -->
                      foo/bosh
              </user-data>

      </data>

      <!-- Specify traceability requirements implemented by this test.

      You can use this for whatever makes sense in your project, but typically they would be defect or user story ids;
      see examples below.
      -->
      <requirement id="UserRequirement_1a"/>
      <requirement id="UserRequirement_2c"/>
      <requirement id="Performance"/>
      <requirement id="UserStory.XYZ-54321"/>

      <!-- NB: Existing tests may nest some of the above elements under elements such as
      description/classification/data/traceability rather than directly under the pysystest node, but these extra
      elements are no longer required (or recommended).
      -->

</pysystest>

See the “cookbook” PySys sample that this XML file comes from to see how these work in practice.

Sample pysysdirconfig.xml

You can provide descriptor defaults for an entire subdirectory of tests by adding a pysysdirconfig.xml file, which avoids the need to copy mode/group/id-prefix information into each individual test, reduces the chance of mistakes, and makes it easier to add new modes and other setting later as your project evolves.

Most of the directory-level options are the same as pysystest.xml but there are a few options that don’t make sense at the directory level, and also additional options for directories such as id-prefix.

<?xml version="1.0" encoding="utf-8"?>
<pysysdirconfig>
      <!-- This file provides default configuration for all tests under this directory.

      Settings are also inherited from pysysdirconfig.xml files in parent directories
      (and the pysysdirconfig element from pysysproject.xml if present).
      -->

Prefix

Specify a prefix to be added to the id of all tests under this directory. This is in addition to any defined by individual tests.

<id-prefix>MyServer.Performance.</id-prefix>

Maker templates

Specify the templates that can be used by “pysys make” when creating new tests. You may wish to provide project or directory-specific template(s) to simplify creating new tests for different purposes, and to encourage test creators to follow best practices (rather than just copying from a random test which may not be a good example).

Run “pysys make -h” to see which templates are available for creating tests under the current directory.

The first template (in the lowest-level pysysdirconfig.xml file) is used by default, but that can be customized by using set-default-maker-template to name a different template (for example at project level or in another pysysdirconfig higher up the directory tree).

Each template is defined by:

• name: A short name used to select this template on the command line (“lowercase-with-hyphens” naming convention).

• description: A brief summary describing what kind of test is created by this template. This text is displayed when running “pysys make -h”.

• copy: A comma-separated list of files and directories to be copied into the test directory, supporting * globs. Either specify a path relative to the directory containing this XML file, or an absolute path using project properties. To use files from the default PySys template use ${pysysTemplatesDir}/test/*. You could store your directory-specific templates in a _pysys_templates/ directory alongside this XML file (with a .pysysignore file to stop them being treated as test cases), or you could use a real (but simple) test to copy from (with suitable regex replacements to make it more generic).

• mkdir: By default empty directories are created for the configured input, output and reference directories. Specify this attribute to define explicitly which empty directories should be created.

• replace: Specify any dynamic substitutions, which will be applied in all copied files. It is recommended to use only ASCII-characters (but if any non-ASCII characters are provided they will be substituted as UTF-8). The replacement “with” expressions can include project properties using ${propname} syntax, or the special strings @{DIR_NAME}, @{USERNAME}, @{DATE}, @{DEFAULT_DESCRIPTOR}. If no replace elements are explicitly provided, the following default replacements are used:

  • @@DATE@@ -> @{DATE} (the current date)

  • @@USERNAME@@ -> @{USERNAME} (the currently logged in user)

  • @@DIR_NAME@@ -> @{DIR_NAME} (the basename of the directory being created, typically the test id)

  • @@DEFAULT_DESCRIPTOR@@ -> @{DEFAULT_DESCRIPTOR} (the __pysys_title__ and other descriptor lines from the standard PySys default test template, after which you can add your customized Python test class)

  • @@LINE_LENGTH_GUIDE@@ -> @{LINE_LENGTH_GUIDE} (80 underline characters to indicate to the user when they are exceeding the recommended maximum length for a test title; this string can be customized for different lengths etc using the pysystestTemplateLineLengthGuide project property)

By default PySys creates .py files with tabs for indentation (as in previous PySys releases). If you prefer spaces, just set the pythonIndentationSpacesPerTab project property to a string containing the required spaces per tab.

See Create new test templates for pysys make for more on how to create templates.

<maker-template name="perf-test" description="a performance test including configuration for my fictional performance tool"
        copy="${pysysTemplatesDir}/default-test/*, ./_pysys_templates/perf/my-perf-config.xml"/>

<maker-template name="my-test" description="a test with the Python code pre-customized to get things started"
        copy="./_pysys_templates/MyTemplateTest/*" />

<maker-template name="foobar-test" description="an advanced test based on the existing XXX test"
        copy="./PySysDirConfigSample/*"
        mkdir="ExtraDir1, ExtraDir2"
>
        <replace regex='__pysys_title__ *= r"""[^"]*"""' with='__pysys_title__   = r""" Foobar - My new @{DIR_NAME} test title TODO """'/>
        <replace regex='__pysys_authors__ *= "[^"]*"'    with='__pysys_authors__ = "@{USERNAME}"'/>
        <replace regex='__pysys_created__ *= "[^"]*"'    with='__pysys_created__ = "@{DATE}"'/>
        <replace regex='@@DIR_NAME@@'                    with='@{DIR_NAME}'/>
</maker-template>

<maker-template name="pysys-xml-test" description="an old-style test with pysystest.xml and run.py"
        copy="${pysysTemplatesDir}/pysystest-xml-test/*"/>

<set-default-maker-template name="my-test"/>

Groups

Specify the groups that all tests under this directory will be tagged with, allowing them to be selected for inclusion/exclusion in test runs. Groups are usually named in camelCase. These groups are in addition to any defined by individual tests, and (if inherit=true) any from parent pysysdirconfigs.

The disableCoverage group is a special group used by code coverage writers to ensure coverage tools are disabled for tests that are performance-critical.

<groups inherit="true" groups="performance, disableCoverage"/>

Modes

Specify the list of modes tests under this directory can be run in.

Like test ids, mode names are usually TitleCase, with multiple dimensions delimited by an _ underscore, e.g. CompressionGZip_AuthNone.

Test modes are configured with a Python lambda that returns a list of modes, using a single parameter which is an instance of pysys.config.descriptor.TestModesConfigHelper providing access to the inherited modes (and other useful functions/fields). Each mode in the returned list is defined by a dictionary containing parameters to be set on the test object and/or a mode name. If the mode name is not explicitly provided, a default mode name is generated by concatenating the parameter values with _ (with a paramName= prefix for any numeric/boolean values).

In your Python lambda you can return a simple list of modes, or combine your own modes with inherited modes defined by parent pysysdirconfigs. You can also use the power of Python list comprehensions to exclude certain modes, perhaps based on dynamic information such as the operating system. Project properties can be accessed using helper.project.PROPERTY_NAME. Avoid expensive operations such as reading the file system from your lambda if possible.

The first listed mode is the “primary” mode, which is the one that is used by default when executing tests with no -m argument.

It’s often useful to combine multiple mode ‘dimensions’, for example all the combinations of a list of web browsers with a list of databases, or compression methods and authentication types. Rather than writing out every combination manually, you can use the function helper.createModeCombinations to automatically generate all combinations.

A test can use self.mode to find out which mode it is executing and/or self.mode.params to access any parameters.

The specified Python code is invoked using pysys.utils.safeeval.safeEval.

<modes>
        lambda helper: [
                mode for mode in
                        helper.createModeCombinations( # Takes any number of mode lists as arguments and returns a single combined mode list
                                helper.inheritedModes,
                                {
                                                'CompressionNone': {'compressionType':None, 'isPrimary':True},
                                                'CompressionGZip': {'compressionType':'gzip'},
                                },
                                [
                                        {'auth':None}, # Mode name is optional
                                        {'auth':'OS'}, # In practice auth=OS modes will always be excluded since MyFunkyOS is a fictional OS
                                ])
                # This is Python list comprehension syntax for filtering the items in the list
                if (mode['auth'] != 'OS' or helper.import_module('sys').platform == 'MyFunkyOS')
        ]
</modes>

<!--
NB: Older (pre-2.0) PySys projects used a more basic and limited syntax for specifying modes:

        <modes inherit="true">
                <mode>CompressionNone</mode>
                <mode>CompressionGZip</mode>
        </modes>
-->

Execution order

Specify as a floating point number an indicator of when to run the tests under this directory, relative to other tests/directories with a higher or lower hint. Empty string hint=”” means inherit. The default priority is 0.0 so set the hint to a higher value to execute tests later, or a negative value to execute tests earlier.

<execution-order hint="+100.0"/>

Skip

Comment/uncomment this to mark all tests under this directory as skipped.

<skipped reason="These tests are skipped until Bug-1234 is fixed"/>

Advanced

By convention the test class uses the pysystest.py module if present or else module=”run.py” located in the test directory, but it is possible to use a different path (even an absolute path). If you want to use a single Python class for lots of tests, omit the module= attribute, ensure the class has at least one level of Python package, and make sure it’s available on the project’s <pythonpath> .

      <class name="PySysTest" module="${testRootDir}/pysys-extensions/MySharedTestClass.py"/>

      <!-- You can customize the Input/Output/Reference directory names if you wish (or even provide an absolute
      paths if needed). These can also be specified using the older names output/input/reference with a path= attribute.
      -->
      <output-dir>MyOutput</output-dir>
      <input-dir>${testRootDir}/pysys-extensions/my_shared_input_files</input-dir>
      <reference-dir>MyReference</reference-dir>

      <data>
              <!-- The ability to add user-defined data to the test descriptor is mostly useful when using a
              shared Python class for lots of tests.

              Project properties (but not other user-data values) can be substituted into the value using ${...},
              and ${eval: xxx} syntax can be used to evaluate some Python code (with project properties as Python variables).

              If a static field of the same name exists on the test class, it will be overridden with the associated user-data
              value, with some basic type coersion to match the default value of the static field where possible.
              -->
              <user-data name="myTestDescriptorData" value="foobar"/>

              <!-- For long values such as paths the value can be specified in a text (or CDATA) node, and if the
              value is to be converted to a list, newline and/or comma can be used as delimiters. -->
              <user-data name="myTestDescriptorPath">
                      foo/foo-${os_myThirdPartyLibraryVersion}
                      foo/bar, foo/baz
                      <!-- Comments and whitespace are ignored when converting a string to a list. -->
                      foo/bosh
              </user-data>
      </data>

      <!-- Specify traceability requirements implemented by all tests under this directory.

      You can use this for whatever makes sense in your project, but typically they would be defect or user story ids;
      see examples below.
      -->
      <requirement id="UserRequirement_1a"/>
      <requirement id="UserRequirement_2c"/>
      <requirement id="Performance"/>
      <requirement id="UserStory.XYZ-54321"/>

      <!-- NB: Existing tests may nest some of the above elements under elements such as
      description/classification/data/traceability rather than directly under the pysystest node, but these extra
      elements are no longer required (or recommended).
      -->

</pysysdirconfig>