Welcome to PySys!

PySys is a powerful cross-platform framework for writing great system/integration tests.

It provides a comprehensive package of methods to make all the common system/integration testing operations a breeze, with all the flexibility and power of the Python language at your fingertips. PySys is a framework that gives you a single unified way to control the selection, ordering and reporting of every type of test including system correctness, performance, soak/robustness testing, unit testing and manual testing.

Whatever language the application you’re testing is written in, and whatever platforms it needs to run on, PySys can help!

Key features include:

  • A comprehensive library of assertion methods appropriate for system-level testing, such as checking for error/success messages in log files and comparing the contents of output files.

  • A comprehensive library of methods to automate platform-independent process starting, orchestration, and cleanup, for both Windows and Unix-based systems. Includes common operations such as:

    • dynamically picking a free TCP/IP port,

    • waiting until a server is running on a specified port,

    • waiting until a file contains a specified message,

    • powerful primitives for pre-processing text files (e.g. configuration input files, or output logs)

    • aborting early with a clear failure messages if an error is detected in a log file

  • Support for executing tests in parallel to significantly speed up execution time, with a flexible mechanism for controlling execution order.

  • Ability to cycle a test many times (and in parallel) to reproduce rare race conditions.

  • Support for executing the same test in several modes during your test run (for example against different web browsers, databases, etc).

  • A process memory monitoring framework to check for memory leaks when soak testing your application.

  • A performance monitoring framework for recording and aggregating latency, throughput and other performance metrics.

  • A pluggable “writers” framework for recording test outcomes in any format, including a standard JUnit-compatible XML results writer and output archiver in the box, and support for running tests under CI providers such as GitHub(R) Actions and Travis CI(R).

  • Integrated support for running PyUnit tests and doctests, in case your application is also written in Python.

  • Integrated support for executing manual/interactively driven test cases.

  • Test categorization and selective include/exclude execution, using per-test classification groups.

  • Support for Windows, Linux, macOS and Solaris.

PySys was created by Moray Grieve. The current maintainer is Ben Spiller. This is a community project so we welcome your contributions, whether enhancement issues or GitHub pull requests!

Note

Use the issue tracker if you want to report a bug or feature request in the PySys test framework. For how-tos and advice, ask a question.