Getting Started¶

After installation, to see the available options to the pysys.py script use:

> pysys.py --help

The script has four main commands:

makeproject to create your top-level testing project configuration file,
make to create individual testcases,
run to execute them, and
clean to delete testcase output after execution.

For detailed information, see the --help command line.

To get started, create a new directory to hold your tests. Then run the makeproject command from that directory to add a pysysproject.xml file which will hold default settings for your tests:

> mkdir test
> cd test
> pysys.py makeproject

Then to create your first test, run:

> pysys.py make MyApplication_001

This will create a MyApplication_001 subdirectory with a pysystest.py file that contains both “descriptor” metadata about the test such as its title, and a Python class where you can add the logic to execute your test, and to validate that the results are as expected.

To run your testcases, simply execute:

> pysys.py run

To give a flavour for what’s possible, here’s a system test for checking the behaviour of a server application called MyServer, which shows of the most common PySys methods:

__pysys_title__   = r""" MyServer startup - basic sanity test (+ demo of PySys basics) """

__pysys_purpose__ = r""" To demonstrate that MyServer can startup and response to basic requests.
  """

class PySysTest(pysys.basetest.BaseTest):
  def execute(self):
    # Ask PySys to allocate a free TCP port to start the server on (this allows running many tests in
    # parallel without clashes)
    serverPort = self.getNextAvailableTCPPort()

    # A common system testing task is pre-processing a file, for example to substitute in required
    # testing parameters
    self.copy(self.input+'/myserverconfig.json', self.output+'/', mappers=[
      lambda line: line.replace('@SERVER_PORT@', str(serverPort)),
    ])

    # Start the server application we're testing (as a background process)
    # self.project provides access to properties in pysysproject.xml, such as appHome which is the
    # location of the application we're testing
    server = self.startProcess(
      command   = self.project.appHome+'/my_server.%s'%('bat' if IS_WINDOWS else 'sh'),
      arguments = ['--configfile', self.output+'/myserverconfig.json', ],
      environs  = self.createEnvirons(addToExePath=os.path.dirname(PYTHON_EXE)),
      stdouterr = 'my_server', displayName = 'my_server<port %s>'%serverPort, background = True,
      )

    # Wait for the server to start by polling for a grep regular expression. The errorExpr/process
    # arguments ensure we abort with a really informative message if the server fails to start
    self.waitForGrep('my_server.out', 'Started MyServer .*on port .*', errorExpr=[' (ERROR|FATAL) '], process=server)

    # Run a test tool (in this case, written in Python) from this test's Input/ directory.
    self.startPython([self.input+'/httpget.py', f'http://localhost:{serverPort}/data/myfile.json'],
      stdouterr='httpget_myfile')

  def validate(self):
    # This method is called after execute() to perform validation of the results by checking the
    # contents of files in the test's output directory. Note that during test development you can
    # re-run validate() without waiting for a full execute() run using "pysys run --validateOnly".

    # It's good practice to check for unexpected errors and warnings so they don't go unnoticed
    self.assertGrep('my_server.out', ' (ERROR|FATAL|WARN) .*', contains=False)

    # Checking for exception stack traces is also a good idea; and joining them into a single line with a mapper will
    # give a more descriptive error if the test fails
    self.assertGrep('my_server.out', r'Traceback [(]most recent call last[)]', contains=False,
      mappers=[pysys.mappers.JoinLines.PythonTraceback()])

    self.assertThat('message == expected',
      message=pysys.utils.fileutils.loadJSON(self.output+'/httpget_myfile.out')['message'],
      expected="Hello world!",
      )

    self.logFileContents('my_server.out')

If you’re curious about any of the functionality demonstrated above, there’s lots of helpful information on these methods and further examples in the documentation:

Now take a look at pysys.basetest to begin exploring more of the powerful functionality PySys provides to help you implement your own pysystest.py system tests.

The sample projects under https://github.com/pysys-test are a great starting point for learning more about PySys, and for creating your first project.