Node:testsuite Invocation, Next:Making testsuite Scripts, Previous:Writing testsuite.at, Up:Using Autotest

Running testsuite Scripts

Autotest test suites support the following arguments:

--help

-h

Display the list of options and exit successfully.

--version

-V

Display the version of the test suite and exit successfully.

--clean

-c

Remove all the files the test suite might have created and exit. Meant for clean Makefile targets.

--list

-l

List all the tests (or only the selection), including their possible keywords.

By default all the tests are performed (or described with --list) in the default environment first silently, then verbosely, but the environment, set of tests, and verbosity level can be tuned:

variable=value

Set the environment variable to value. Do not run FOO=foo ./testsuite as debugging scripts would then run in a different environment.

The variable AUTOTEST_PATH specifies the testing path to prepend to PATH. It handles specially relative paths (not starting with /): they are considered to be relative to the top level of the package being built. All the directories are made absolute, first starting from the top level build tree, then from the source tree. For instance ./testsuite AUTOTEST_PATH=tests:bin for a /src/foo-1.0 source package built in /tmp/foo results in /tmp/foo/tests:/tmp/foo/bin and then /src/foo-1.0/tests:/src/foo-1.0/bin being prepended to PATH.

number

number-number

number-

-number

Add the corresponding test groups, with obvious semantics, to the selection.

--keywords=keywords

-k keywords

Add to the selection the test groups which title or keywords (arguments to AT_SETUP or AT_KEYWORDS) match all the keywords of the comma separated list keywords.

Running ./testsuite -k autoupdate,FUNC will select all the tests tagged with autoupdate and FUNC (as in AC_CHECK_FUNC, AC_FUNC_FNMATCH etc.) while ./testsuite -k autoupdate -k FUNC runs all the tests tagged with autoupdate or FUNC.

--errexit

-e

If any test fails, immediately abort testing. It implies --debug: post test group clean up, debugging script generation, and logging are inhibited. This option is meant for the full test suite, it is not really useful for generated debugging scripts.

--verbose

-v

Force more verbosity in the detailed output of what is being done. This is the default for debugging scripts.

--debug

-d

Do not remove the files after a test group was performed --but they are still removed before, therefore using this option is sane when running several test groups. Do not create debugging scripts. Do not log (in order to preserve supposedly existing full log file). This is the default for debugging scripts.

--trace

-x

Trigger shell tracing of the test groups.