Project Configuration ===================== Each PySys project has a configuration file called ``pysysproject.xml`` at the top level. This file contains ``property`` elements for any user-defined properties that will be used by your tests such as credentials, server names, which are available to your tests using `self.project `. The project file also contains allows customization of how PySys executes tests and reports on the results. When starting a new PySys test project you should create a minimal project file from scratch using ``pysys makeproject``, then add any additional elements you require. To illustrate the range of available elements, here is a large project file that demonstrates all the main functionality. Header ~~~~~~ .. code-block:: xml 2.2 3.8 Project properties ~~~~~~~~~~~~~~~~~~ The following standard project properties are always defined and can be accessed through ${prop} syntax: * ``${testRootDir}`` - Path of the root directory containing the pysysproject.xml file * ``${outDirName}`` - The basename (with parent dirs removed) from the outdir for this test run. This may be the name of the current OS or a unique user-specified name for the test run. * ``${os}`` - The operating system name e.g. ``windows``, ``linux``, ``darwin``. * ``${osfamily}`` - The operating system family, either ``windows`` or ``unix``. * ``${startDate}`` - The date that this test run was started, in a form that can be used in filenames. * ``${startTime}`` - The (local) time that this test run was started, in a form that can be used in filenames. * ``${hostname}`` - The (non-qualified) name of the host this is running on, suitable for including in filenames. * ``${username}`` - The (lowercase) username of the account executing PySys. * ``${/}`` - The backslash or forward-slash character used on this OS. Python expressions can be evaluated using ``${eval: ...}`` (and can use project properties as Python variables, and/or from a dictionary named ``properties``). In addition, within this file ``${env:VARNAME}`` (or ``${env.VARNAME}``) syntax can be used to access environment variables. Project properties can be used as substitution variables within the project file, and are set as attributes on the `pysys.config.project.Project` class for use by tests - so in the example below, the ``appHome`` property would be available to tests as ``self.project.appHome`` If a property value contains any properties or environment variables that do not exist, the ``default`` is used instead (or "" if a default is not explicitly specified). If the default also contains undefined properties the project will fail to load. The ``pathMustExist=true`` attribute can be used to stop the project loading if the property specifies a path that does not exist. Use ``path=`` instead of ``value=`` if the value is a path, to automatically normalize ``/`` and ``.`` characters. .. code-block:: xml Special project properties ~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: xml Override this to change the default behaviour for when PySys will log absolute rather than current dir relative paths in situations such as failure outcomes and test outcome summary. It is often more useful to get absolute paths when running tests within an IDE that supports clicking on failures to jump to them. If the ``PYSYS_LOG_ABSOLUTE_PATHS`` environment variable is set it will override any value set using the project property. .. code-block:: xml Configuring plugins ~~~~~~~~~~~~~~~~~~~ See also :ref:`pysys/UserGuide:Runner and writer plugins`. If you want to provide some custom plugins/extensions you need to add them to the PYTHONPATH used by PySys as shown below. Usually the specified directory should contain a Python module named after your organization and containing an empty __init__.py file to mark it as a Python package (such as ``pysys-extensions/myorg/__init__.py``). .. code-block:: xml Runner plugins are classes that are instantiated when the BaseRunner performs its setup() at the beginning of a test run. They can provide extra functionality both at the beginning of a test run, and also (by calling `addCleanupFunction() ` from ``setup()``) at the end after testing has finished. For example, a runner plugin could be used to add support for starting a database server or virtual machine to be shared by all tests. Each plugin class must provide a ``setup(self, runner)`` method that accepts the owner runner (BaseRunner instance) as a parameter. Any configuration properties are assigned as attributes before setup is called. The plugin instance can optionally be assigned as a field of the runner using the specified "alias" so that its methods and fields are available for use by tests. The alias can be any valid Python identifier but must not conflict with other plugins or fields that PySys sets on the BaseRunner; usually a brief lowercase name identifying your organization or the purpose of the plugin is best. .. code-block:: xml For advanced cases it is possible to provide a custom `pysys.baserunner.BaseRunner` subclass. However, in most cases this isn't necessary so always consider whether the composition "runner-plugin" approach would do the job before using runner inheritance. .. code-block:: xml WARNING: The "test-plugin" paradigm is no longer recommended - instead use a "helper" class inherited in the individual tests that need the functionality rather than in the project XML. Test plugins are additional classes instantiated when each test's BaseTest is instantiated, and accessible to testcases (using the specified alias) as a field of the BaseTest. They provide a way to expose extra functionality for use by your testcases, for example support for additional languages and technologies. Each plugin class must provide a ``setup(self, testObj)`` method that accepts the owner testObj (BaseTest instance) as a parameter. Any configuration properties are assigned as attributes before setup is called. Each plugin class must provide a constructor that accepts the parent testObj (BaseTest instance) as a parameter. The plugin instance is assigned as a field of the test object using the specified "alias" so that its methods and fields are available for use. The alias can be any valid Python identifier but must not conflict with other plugins or fields that PySys sets on the BaseTest; usually a brief lowercase name identifying your organization or the purpose of the plugin is best. .. code-block:: xml Writers ~~~~~~~ Configures the writers that implement reporting of test outcomes, typically to disk, to the console, or to a CI system. For full details of the configuration properties of each writer, and the API for creating custom writers, see `pysys.writer` in the API reference. Each writer element specifies the classname (which should be available on the pythonpath). .. code-block:: xml lambda testObj: 'unitTest' in testObj.descriptor.groups or testObj.project.getProperty('isLocalDeveloperTestRun',False) Performance reporters ~~~~~~~~~~~~~~~~~~~~~ By default PySys will record performance results using a default instance of `pysys.perf.reporters.CSVPerformanceReporter`. However if desired you can configure an alternative list of reporters such as `pysys.perf.reporters.JSONPerformanceReporter` or custom reporters of your own. .. code-block:: xml Configuration for how tests execute ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: xml Test descriptor defaults ~~~~~~~~~~~~~~~~~~~~~~~~ A project file can contain a ``pysysdirconfig`` element to configure directory-level defaults that will apply to all test descriptors. See the :ref:`pysys/TestDescriptors:Sample pysysdirconfig.xml` documentation for more information about the available options, including how to customize how new tests are created using maker templates. .. code-block:: xml . Output Reference For advanced cases, use a custom maker class for constructing new testcases. Custom maker classes can extend from the `pysys.launcher.console_make.DefaultTestMaker` class in order to customize details such as the automatic validation of new test ids. .. code-block:: xml PySys run.log and console formatters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In most projects there is no reason to customize this, but it can be used if you real want a different log line format (incl date/time format) or console coloring. Specify custom formatters for logging to the console or run.log, and/or configure the formatter appropriately through custom properties. Custom formatters should be defined using the classname and module attributes and should extend the `pysys.utils.logutils.BaseLogFormatter` class. If no classname and module is given the default `pysys.utils.logutils.ColorLogFormatter` is assumed. The ColorLogFormatter allows specification of the message and date strings using the messagefmt and datafmt attributes. Enabling color to the console (stdout) formatter can be done using the color property, and the colors for supported message types can be specified via the color: property. See below for more details for the default color types and categories. Supported colors are BLUE, GREEN, YELLOW, RED, MAGENTA, CYAN, WHITE and BLACK. .. code-block:: xml Project-specific help information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: xml My project ---------- -Xiterations=N Override the number of request/response iterations performed during execution of each performance test. The application being tested is at "${appHome}"; to override use ${$}{PYSYS_APP_HOME}. Don't forget to execute the MyProject build before running the tests!