Project ConfigurationΒΆ

Each PySys project has a configuration file called pysysproject.xml at the top level. This file contains property elements for any user-defined properties that will be used by your tests such as credentials, server names, which are available to your tests using self.project. The project file also contains allows customization of how PySys executes tests and reports on the results.

When starting a new PySys test project you should create a minimal project file from scratch using pysys makeproject, then add any additional elements you require.

To illustrate the range of available elements, here is a large project file that demonstrates all the main functionality:

<?xml version="1.0" encoding="utf-8"?>

      Specify a minimum required pysys version to run these tests

      Specify a minimum required python version to run these tests

              The following standard project properties are always defined and can be accessed through ${prop} syntax:

                      ${testRootDir} - Path of the root directory containing the pysysproject.xml file
                      ${outDirName}  - The basename (with parent dirs removed) from the outdir for this test run.
                                       This may be the name of the current OS or a unique user-specified name for the test run.
                      ${os}          - The operating system name e.g. 'windows', 'linux', 'darwin'.
                      ${osfamily}    - The operating system family - 'windows' or `unix`.
                      ${startDate}   - The date that this test run was started, in a form that can be used in filenames.
                      ${startTime}   - The (local) time that this test run was started, in a form that can be used in filenames.
                      ${hostname}    - The (non-qualified) name of the host this is running on, suitable for including in filenames.

              In addition, within this file ${env.VARNAME} syntax can be used to access environment variables.

      <!-- Property identifying the home directory of the application build being tested.

      Binaries and configuration files can be specified relative to this directory
      to avoid having to hardcode locations inside each individual testcase.

      A different directory can be used temporarily by setting environment variable PYSYS_APP_HOME.

      Project properties can be used as substitution variables within the project file, and
      are set as attributes on the Project class for use by tests. In the example below, "appHome"
      would be available to tests as "self.project.appHome".

      If a property value contains any properties or environment variables that do not exist,
      the "default" is used instead (or "" if a default is not explicitly specified). If the default
      also contains undefined properties the project will fail to load.

      <property name="appHome" value="${env.PYSYS_APP_HOME}" default="${testRootDir}/.." pathMustExist="true"/>

      <!-- If provided, the specified command line will be executed (in testRootDir) to populate the vcsCommit field
              in runner.runDetails with the current commit number from your version control system. -->
      <property name="versionControlGetCommitCommand" value="git show -s --format=%h"/>

      Set default LANG for child processes on Unix.
      Useful for making all machines behave the same. Ensure that the specified
      locale is installed on all machines.
      <!-- <property name="defaultEnvironsDefaultLang" value="en_US.UTF-8"/> -->

      The defaultEnvirons. prefix can be used to provide a default environment variable for use by
      all processes that PySys starts. For example, to set Java(R)'s JAVA_TOOL_OPTIONS environment variable:
      <property name="defaultEnvirons.JAVA_TOOL_OPTIONS" value="-Xmx512M"/>

      Import project properties from a file, using the specified prefix for namespacing.

      If pathMustExist=true, the project will fail to load if the specified file does not
      exist. If pathMustExist=false the project will silently ignore a missing properties file,
      which can be useful when using this feature to provide optional user-specific overrides.

      The properties file should be of the format name=value (one pair specified per line).
      Each imported property line is handled just the same as a standard "property" element,
      so for example other project properties can be referenced withint the values
      using ${...} syntax.

      The set of property keys that are imported can be filtered with a regular expression using
      the includes="regex" and excludes="regex" optional attributes.
      <property file="pysys-${osfamily}.properties" pathMustExist="false" prefix="os_"/>

      Add a path to the python path to allow importing custom modules, e.g. extensions to the
      BaseTest or BaseRunner classes, or customer test output writers.
      <pythonpath value="${testRootDir}/pysys-extensions" />

      <!-- Test plugins are additional classes instantiated when each test's BaseTest is instantiated, and accessible
      to testcases (using the specified alias) as a field of the BaseTest. They provide a way to expose extra
      functionality for use by your testcases, for example support for additional languages and technologies.

      Each plugin class must provide a constructor __init__(self, testObj, pluginProperties) that accepts the owner
      testObj (BaseTest instance) as a parameter and a dictionary of plugin properties provided as attributes
      and/or <property> elements.

      Each plugin class must provide a constructor that accepts the parent testObj (BaseTest instance) as a parameter.
      The plugin instance is assigned as a field of the test object using the specified "alias" so that its
      methods and fields are available for use. The alias can be any valid Python identifier but must not conflict
      with other plugins or fields that PySys sets on the BaseTest; usually a brief lowercase name
      identifying your organization or the purpose of the plugin is best. -->
              <test-plugin classname="myorg.plugins.MyTestPlugin" alias="myorg">
                      <property name="myProp" value="..."/>

      <!-- Runner plugins are classes that are instantiated when the BaseRunner performs its setup() at
      the beginning of a test run. They can provide extra functionality both at the beginning of a test
      run, and also (by calling addCleanupFunction from setup()) at the end after testing has finished. For example,
      a runner plugin could be used to add support for starting a database server or virtual machine to be shared
      by all tests, or to collect.

      Each plugin class must provide a constructor __init__(self, runner, pluginProperties) that accepts the owner
      runner (BaseRunner instance) as a parameter and a dictionary of plugin properties provided as attributes
      and/or <property> elements.

      The plugin instance can optionally be assigned as a field of the runner using the specified "alias" so that its
      methods and fields are available for use by tests. The alias can be any valid Python identifier but must not
      conflict with other plugins or fields that PySys sets on the BaseRunner; usually a brief lowercase name
      identifying your organization or the purpose of the plugin is best. -->
              <runner-plugin classname="myorg.plugins.MyRunnerPlugin" alias="myorg">
                      <property name="myProp" value="..."/>

      For advanced cases it is possible to provide a custom BaseRunner subclass. However, consider whether the
      composition "runner-plugin" approach would do the job before using runner inheritance.

      For example, to use the MyRunner class in the myorg module use:
              <runner classname="myorg.MyRunner"/>

      Use a custom maker class for constructing new testcases. Custom maker classes can extend from the
      ConsoleMakeTestHelper class in order to create templated scripts e.g. when a test module has
      a repeatable pattern for running or validating etc.
      <maker classname="myorg.MyMaker"/>

      Configures the writers that implement reporting of test outcomes, typically to disk, to the console, or
      to a CI system.

      For full details of the configuration properties of each writer, and the API for creating custom writers,
      see `pysys.writer` in the API reference.

      The writer element specifies the module (which should be available on the pythonpath) and writer classname.
              <!-- This writer is useful for creating zip archives of failed test output directories when
                      running in record mode on a machine where it is not otherwise easy to access the output
                      directories. The destDir could then be uploaded to a CI system (some CI writers
                      implement this automatically) or manually copied to a file share.
              <writer classname="pysys.writer.testoutput.TestOutputArchiveWriter">
                      <property name="destDir" value="__pysys_output_archives.${outDirName}/"/>
                      <property name="maxTotalSizeMB" value="1024.0"/>
                      <property name="maxArchiveSizeMB" value="200.0"/>
                      <property name="maxArchives" value="50"/>

              <!-- CI writers automatically enable themselves only if running under
                      the associated CI environment.
              <writer classname=""></writer>
              <writer classname=""></writer>

              Add in the JUnit results writer for PySys test output to be written in the widely-used Apache Ant JUnit XML format.

              Use the outputDir property to define the output directory for the JUnit test summary files (the writer will
              produce one file per test into this output directory).
              <writer classname="pysys.writer.outcomes.JUnitXMLResultsWriter">
                      <property name="outputDir" value="__pysys_junit_xml"/>

              <writer classname="pysys.writer.outcomes.XMLResultsWriter" file="__pysys_testsummary_${outDirName}_%Y-%m-%d_%H.%M.%S.xml">
                      Set properties on the XML test output writer class. The available properties that
                      can be set are the stylesheet location, whether to use file URLs in all references
                      to resources on the local disk, and the directory to write the output file (defaults
                      to the current working directory). Note that Mozilla Firefox requires the stylesheet
                      to be located next to the XML file when loading the file, and all references to local
                      resources must be as file URLs. Internet Explorer and Chrome can load the stylesheet
                      from any location on the local disk, but cannot load resources when referenced by a
                      file URL.

                      The filename template is processed through time.strftime so that time information can
                      be set into the filename, e.g. a filename template of 'testsummary.%Y-%m-%d_%H.%M.%S' will result
                      in a file created with a name of  testsummary_2008-10-25_21.33.08.xml etc.

                      <property name="outputDir" value="${testRootDir}"/>
                      <property name="stylesheet" value="./pysys-log.xsl"/>
                      <property name="useFileURL" value="true"/>

              Add in the test results writer if output to a text file is required

              <writer classname="pysys.writer.outcomes.TextResultsWriter" file="__pysys_testsummary_%Y-%m-%d_%H.%M.%S.log">
                      <property name="outputDir" value="${testRootDir}"/>

              Add in the CSV results writer if CSV text output is required. This outputs the test results
              in a column separated list, with headings id, title, cycle, startTime, duration, outcome

              <writer classname="pysys.writer.outcomes.CSVResultsWriter" file="__pysys_testsummary_%Y-%m-%d_%H.%M.%S.csv">
                      <property name="outputDir" value="${testRootDir}"/>

              <!-- Code coverage for .py files -->
              <writer classname="pysys.writer.testoutput.PythonCoverageWriter">
                      <property name="destDir" value="__coverage_python.${outDirName}"/>
                      <property name="pythonCoverageArgs" value="--rcfile=${testRootDir}/internal/utilities/python_coveragerc"/>

              <!-- The ConsoleSummaryResultsWriter displays a summary of non-passed outcomes at the end of the test run,
                      optionally including outcome reason. The ConsoleSummaryResultsWriter is automatically added to the writers
                      list if no other "summary" writer is explicitly configured.
              <writer classname="pysys.writer.console.ConsoleSummaryResultsWriter">
                      <property name="showTestTitle" value="false"/>

              Specify the file encoding to be used for reading/writing text files.

              The first pattern that matches is used to determine the encoding. The pattern is a glob-style expression to be
              matched case-insensitively against either the full path or the basename using Python's fnmatch.fnmatch method.

              The defaults specified here can be overridden or added to by the runner or basetest getDefaultFileEncoding()
              method. See pysys.process.user.ProcessUser.getDefaultFileEncoding for more details.

              <default-file-encoding pattern="run.log" encoding="utf-8"/>

              <default-file-encoding pattern="*.xml" encoding="utf-8"/>
              <default-file-encoding pattern="*.json" encoding="utf-8"/>
              <default-file-encoding pattern="*.yaml" encoding="utf-8"/>


      <execution-order secondaryModesHintDelta="+100.0">
              The execution-order elements provide a way to globally change the ordering hints specified on individual
              tests by adding or subtracting a value from the hints specified on test descriptors in a specified group
              and/or mode.

              Groups and modes can be identified with a full name or a regular expression.

              The secondaryModesHintDelta value is used to order tests so that all tests execute in their primary
              mode before any secondary modes are executed. The 2nd mode (the one following the primary mode) has its
              execution order hint incremented by secondaryModesHintDelta, the third by 2*secondaryModesHintDelta etc,
              which ensures the modes are spaced out. To disable this behaviour and execute all modes of each test
              before moving on to the next test set it to 0.0. If not specified, the default value is +100.0.

              <execution-order hint="+20.0" forMode="MyMode_.*"/>
              <execution-order hint="-10.5" forGroup="performance" forMode="MyMode"/>

      In most projects there is no reason to provide a <formatters> element, but it can be used if you wish to customize
      the log line format (incl date/time format) or console coloring.

      Specify custom formatters for logging to the console or run.log, and/or configure the formatter
      appropriately through custom properties. Custom formatters should be defined using the classname
      and module attributes and should extend the pysys.utils.logutils.BaseLogFormatter class. If no
      classname and module is given the default pysys.utils.logutils.ColorLogFormatter is assumed.

      The ColorLogFormatter allows specification of the message and date strings using the messagefmt and
      datafmt attributes. Enabling color to the console (stdout) formatter can be done using the color
      property, and the colors for supported message types can be specified via the color:<category> property.
      See below for more details for the default color types and categories. Supported colors are BLUE,
              <formatter classname="pysys.utils.logutils.ColorLogFormatter"
                              name="stdout" messagefmt="%(asctime)s %(levelname)-5s %(message)s" datefmt="%H:%M:%S">
                      <property name="color" value="true"/>
                      <property name="color:warn" value="MAGENTA"/>
                      <property name="color:error" value="RED"/>
                      <property name="color:traceback" value="RED"/>
                      <property name="color:debug" value="BLUE"/>
                      <property name="color:filecontents" value="BLUE"/>
                      <property name="color:details" value="CYAN"/>
                      <property name="color:outcomereason" value="CYAN"/>
                      <property name="color:progress" value="CYAN"/>
                      <property name="color:performance" value="CYAN"/>
                      <property name="color:timed out" value="MAGENTA"/>
                      <property name="color:failed" value="RED"/>
                      <property name="color:passed" value="GREEN"/>
                      <property name="color:skipped" value="YELLOW"/>

              <formatter classname="pysys.utils.logutils.BaseLogFormatter"
                              name="runlog" messagefmt="%(asctime)s %(levelname)-5s %(message)s" datefmt=""/>

              <!-- Add project-specific text to be appended to the "pysys run -h".

              You can use ${...} project properties, or ${$} to escape the $ character.