Integration test suite¶
At the time of writing this document, we have 60 integration tests which tests abrt, libreport and satyr functionality in real world scenarios.
These tests are run nightly for each of the currently supported releases of Fedora and Red Hat Enterprise Linux. You can find the results on our public mirror.
Running the test suite¶
Don’t run the test suite on your machine directly, use virtual machine or machine dedicated for the testing.
- In your virtual machine or dedicated machine:
- clone the ABRT repository:
git clone git://git.fedorahosted.org/abrt.git
- go to
- clone the ABRT repository:
This will run all the tests in order specified in
./aux/test_order. When the testing
is done, the output can be found in
To override the destination
directory or the other options, you can edit
Running single test¶
To run only one of the tests, you have to run it via
This script prepares your system by restarting required services
and cleaning up possible stale files.
For example, to run
dbus-api test, you have to pass the
whole path to
Running a subset of the tests¶
To run only part of the tests, edit
file and comment out the tests you don’t want to run.
For example, when using the test suite to test already installed packages,
you don’t need to build and install git versions.
In this case, you can comment out
and all the
\*-make-check tests. These are now disabled by default.
Writing new integration test¶
- To create a new test you have to:
- create a directory reflecting its name
- create two files in that directory:
- add the test to
It’s always a good idea to copy one of the existing tests as a base for your test. Good candidates are:
run-abrtdwhich basically serves as an example,
ccpp-pluginif you need to test crash handling (if you require crash directory),
dbus-apiif you need to test DBus functionality.
Please provide an explanation for complex tests. Either explain certain sections of the
test in comments in
runtest.sh or create
README file inside the directory of the test.
Output directory structure¶
To avoid painful digging in several distinct files logging is now divided into several logical levels:
$OUTPUT_ROOT variable is defined in
aux/config.sh which represents the root
for all logs.
For each stage of the test suite run there is corresponding directory in root directory. It contains stage.log file which contains output of script governing the stage.
/tmp/abrt-testsuite ├── format/ ├── post/ ├── pre/ ├── report/ ├── results └── test/
- install required packages, collect log files, collect system information
- run all the tests
- format the results for REPORT stage
- report the results
- collect logs, cleanup
TEST stage there is an additional subdirectory for each test case:
/tmp/abrt-testsuite/test/ ├── abrt-make-check ├── abrt-nightly-build ├── abrt-should-return-rating-0-on-fail ├── blacklisted-package ...
Each directory contains several files:
/tmp/abrt-testsuite/test/systemd-init/ ├── dmesg ├── avc ├── fail.log ├── full.log ├── messages └── protocol.log
full.log is mandatory. It contains stdout and stderr of the test run.
protocol.log only contains the protocol generated by BeakerLib. If the test
fails with FATAL error,
protocol.log is not generated. In case of other failures,
these are extracted to
fail.log along with line numbers pointing to lines in
avc each contains log file messages written during the test run.