pysys.perf.reporters

Contains the built-in reporter classes.

CSVPerformanceReporter

class pysys.perf.reporters.CSVPerformanceReporter(project, summaryfile, testoutdir, runner, **kwargs)

Bases: pysys.perf.api.BasePerformanceReporter

Performance reporter which writes to a CSV file.

This reporter writes to a UTF-8 file of comma-separated values that is both machine and human readable and easy to view and use in any spreadsheet program, and after the columns containing the information for each result, contains comma-separated metadata containing key=value information about the entire run (e.g. hostname, date/time, etc), and (optionally) associated with each individual test result (e.g. test mode etc). The per-run and per-result metadata is not arranged in columns since the structure differs from row to row.

After tests have run, the summary file is published with category CSVPerformanceReport using the pysys.writer.api.ArtifactPublisher interface.

The following properties can be set in the project configuration for this reporter:

summaryFile = ''

The filename pattern used for the summary file(s); see DEFAULT_SUMMARY_FILE.

For compatibility purposes, if not specified explicitly, the summary file for the CSVPerformanceReporter can be configured with the project property csvPerformanceReporterSummaryFile, however this is deprecated. This property can also be accessed and configured under the alternative capitalization summaryfile, however this is discouraged as of PySys 2.1+, where summaryFile is the preferred name.

aggregateCycles = False

Enable this if you want PySys to rewrite the summary file at the end of a multi-cycle test with an aggregated file containing the mean and standard deviation for all the cycles, rather than a separate line for each cycle. This may be easier to consume when triaging performance results and looking for regressions.

New in version 2.1.

publishArtifactCategory = 'CSVPerformanceReport'

If specified, the output file will be published as an artifact using the specified category name.

New in version 2.1.

DEFAULT_SUMMARY_FILE = '__pysys_performance/${outDirName}_${hostname}/perf_${startDate}_${startTime}.${outDirName}.csv'

The default summary file if not overridden by the csvPerformanceReporterSummaryFile project property, or the summaryFile= attribute. See getRunSummaryFile(). This is relative to the runner output+’/..’ directory (typically testRootDir, unless --outdir is overridden).

JSONPerformanceReporter

class pysys.perf.reporters.JSONPerformanceReporter(project, summaryfile, testoutdir, runner, **kwargs)

Bases: pysys.perf.api.BasePerformanceReporter

Performance reporter which writes to a JSON file.

After tests have run, the summary file is published with category JSONPerformanceReport using the pysys.writer.api.ArtifactPublisher interface.

New in version 2.1.

The following properties can be set in the project configuration for this reporter:

summaryFile = ''

The .json filename pattern used for the summary file(s); see DEFAULT_SUMMARY_FILE.

publishArtifactCategory = 'JSONPerformanceReport'

If specified, the output file will be published as an artifact using the specified category name.

DEFAULT_SUMMARY_FILE = '__pysys_performance/${outDirName}_${hostname}/perf_${startDate}_${startTime}.${outDirName}.json'

The default summary file if not overridden by the summaryFile= attribute. See getRunSummaryFile(). This is relative to the runner output+’/..’ directory (typically testRootDir, unless --outdir is overridden).

PrintSummaryPerformanceReporter

class pysys.perf.reporters.PrintSummaryPerformanceReporter(project, summaryfile, testoutdir, runner, **kwargs)

Bases: pysys.perf.api.BasePerformanceReporter

Performance reporter which logs a human-friendly summary of all performance results to the console at the end of the test run.

By setting the BASELINES_ENV_VAR environment variable, this reporter will also print out an automatic comparison from the named baseline file(s) to the results from the current test run. This feature is very useful when comparingdifferent strategies for optimizing your application.

New in version 2.1.

BASELINES_ENV_VAR = 'PYSYS_PERFORMANCE_BASELINES'

Set this environment variable to a comma-separated list of performance (e.g. .csv) files to print out an automatic comparison from the baseline file(s) to the results from the current test run.

This feature is very useful when comparing different strategies for optimizing your application.

For best results, use multiple cycles for all test runs so that standard deviation can be calculated.

The filenames can be absolute paths, glob paths using * and **, or relative to the testRootDir. For example:

export PYSYS_PERFORMANCE_BASELINES=__pysys_performance/mybaseline*/**/*.csv,__pysys_performance/optimization1*/**/*.csv