16.2 Writing `testsuite.at'

The `testsuite.at' is a Bourne shell script making use of special Autotest M4 macros. It often contains a call to AT_INIT nears its beginning followed by one call to m4_include per source file for tests. Each such included file, or the remainder of `testsuite.at' if include files are not used, contain a sequence of test groups. Each test group begins with one call to AT_SETUP, it contains an arbitrary number of shell commands or calls to AT_CHECK, and it completes with one call to AT_CLEANUP.

Macro: AT_INIT ([name])

Initialize Autotest. Giving a name to the test suite is encouraged if your package includes several test suites. In any case, the test suite always displays the package name and version. It also inherits the package bug report address.

Macro: AT_TESTED (executables)

Log the path and answer to `--version' of each program in space-separated list executables. Several invocations register new executables, in other words, don't fear registering one program several times.

Autotest test suites rely on the PATH to find the tested program. This saves from generating the absolute paths to the various tools, and makes it possible to test installed programs. Therefore, knowing what programs are being exercised is crucial to understand some problems in the test suite itself, or its occasional misuses. It is a good idea to also subscribe foreign programs you depend upon, to ease incompatibility diagnostics.

Macro: AT_SETUP (test-group-name)

This macro starts a group of related tests, all to be executed in the same subshell. It accepts a single argument, which holds a few words (no more than about 30 or 40 characters) quickly describing the purpose of the test group being started.

Macro: AT_KEYWORDS (keywords)

Associate the space-separated list of keywords to the enclosing test group. This makes it possible to run "slices" of the test suite. For instance if some of your test groups exercise some `foo' feature, then using `AT_KEYWORDS(foo)' lets you run `./testsuite -k foo' to run exclusively these test groups. The title of the test group is automatically recorded to AT_KEYWORDS.

Several invocations within a test group accumulate new keywords. In other words, don't fear registering several times the same keyword in a test group.

Macro: AT_CLEANUP

End the current test group.

Macro: AT_DATA (file, contents)

Initialize an input data file with given contents. Of course, the contents have to be properly quoted between square brackets to protect against included commas or spurious M4 expansion. The contents ought to end with an end of line.

Macro: AT_CHECK (commands, [status = ``0''], [stdout], [stderr])

Execute a test by performing given shell commands. These commands should normally exit with status, while producing expected stdout and stderr contents. If commands exit with status 77, then the whole test group is skipped.

The commands must not redirect the standard output, nor the standard error.

If status, or stdout, or stderr is `ignore', then the corresponding value is not checked.

The special value `expout' for stdout means the expected output of the commands is the content of the file `expout'. If stdout is `stdout', then the standard output of the commands is available for further tests in the file `stdout'. Similarly for stderr with `expout' and `stderr'.