Welcome to PySys!¶
PySys is a powerful cross-platform framework for writing great system/integration tests.
It provides a comprehensive package of methods to make all the common system/integration testing operations a breeze, with all the flexibility and power of the Python language at your fingertips. PySys is a framework that gives you a single unified way to control the selection, ordering and reporting of every type of test including system correctness, performance, soak/robustness testing, unit testing and manual testing.
Whatever language the application you’re testing is written in, and whatever platforms it needs to run on, PySys can help!
Key features include:
A comprehensive library of assertion methods appropriate for system-level testing, such as checking for error/success messages in log files and comparing the contents of output files.
A comprehensive library of methods to automate platform-independent process starting, orchestration, and cleanup, for both Windows and Unix-based systems. Includes common operations such as:
dynamically picking a free TCP/IP port,
waiting until a server is running on a specified port,
waiting until a file contains a specified message,
powerful primitives for pre-processing text files (e.g. configuration input files, or output logs)
aborting early with a clear failure messages if an error is detected in a log file
Support for executing tests in parallel to significantly speed up execution time, with a flexible mechanism for controlling execution order.
Ability to cycle a test many times (and in parallel) to reproduce rare race conditions.
Support for executing the same test in several modes during your test run (for example against different web browsers, databases, or for writing parameterized tests). Python expressions give the power to easily create complex and dynamic lists of modes that combine multi-dimensional sets of parameters.
A performance monitoring framework for recording and aggregating latency, throughput and other performance metrics.
A process memory monitoring framework to monitor memory usage when soak testing your application.
A pluggable “writers” framework for recording test outcomes in any format. Includes a test output archiver, a writer for the ubiquitous JUnit/Ant(TM) XML file format, and built-in support for running tests under CI providers such as GitHub(R) Actions and Travis CI(R).
Integrated support for running PyUnit tests and doctests, in case your application is also written in Python.
Integrated support for executing manual/interactively driven test cases.
Test categorization and selective include/exclude execution, using per-test classification groups.
Support for Windows, Linux and macOS.
PySys was created by Moray Grieve. The maintainer is now Ben Spiller. This is a community project so we welcome your contributions, whether enhancement issues or GitHub pull requests!
Note
Use the issue tracker if you want to report a bug or feature request in the PySys test framework. For how-tos and advice, ask a question.