Documentation for 3.10
Other versions: trunk  4.3  4.0  3.27  Older...

Guide to the TextTest Static GUI
Examining and maintaining the test suites
How to start the Static GUI
The static GUI is started by default unless you specified otherwise in your config file. If you did, it can be started via the command line option "-gx".



As can be seen, the structure of the static GUI is similar to that of the dynamic GUI, and tests can be viewed in much the same way. On the left is a colour-coded tree view of the whole test suite.
The above example shows the same test suite and the same test as previously being viewed in the static GUI. The files can be double clicked in a similar way, here they will invariably be viewed with “view_program” (xemacs/notepad by default). Note this setting can also be configured per result type as described above.
The static GUI can be used to creating new tests or test suites. By clicking a test suite, and filling in the forms marked "Adding Suite" and "Adding Test" (and pressing the corresponding button when complete), the test suite can be extended without the need to create directories and edit files by hand. Note that you can also edit the test suite file from this view, and re-ordering of the tests performed this way will show up in the GUI without needing to restart it.
Running Tests from the Static GUI
In order to run tests, you first need to select some tests to run. You can do this by single-clicking tests in the test window on the left. Use ctrl + left click to build up a multiple selection one test at a time, shift + left click to select everything between what is currently selected and the line clicked, ctrl + A to select everything. Alternatively, you can select tests according to search criteria using the “Select” button and “Selection” tab on the right (see below for details of what can be done).
At the top right is a tab called “Running” which will have three sub-tabs. The tabs “Basic” and “Advanced” can be used to configure a multitude of things about how the tests will be run. At the start the default options should be sufficient. (Note that the tabs are essentially a graphical representation of all command line options that can be given to the dynamic GUI)
Once you are happy with these, press “Run” (on the toolbar or in one of the above tabs). This will start the dynamic GUI on the selected tests.
Selecting tests via search criteria: the static GUI “Selection” tab
On the right there is a “Selection” tab which has a sub-tab “Select Tests” (it should be visible when you start TextTest). This provides a simple search mechanism for finding tests, useful when the test suite grows too large to always select comfortably which tests you want to run via the GUI alone. When the “Select” button is pressed, all tests will be selected which fulfil all of the criteria specified by the text boxes in the “Select” tab. It follows that if no filters are provided and “Select” pressed, all tests will be selected.
There are four “modes” for selection represented by radio buttons at the bottom. “Discard”, the default, will ignore what is already selected. “Extend” will keep the current selection and add to it. “Refine” will match only tests that were already selected and match the search criteria, while “Exclude” will match only test that were not already selected.
Note that the number of selected tests (and the total number of tests) is displayed in the column header of the test view at all times. The various selection criteria can also be tried out from the command line, using the plugin script “default.CountTest”.
Description of all test selection filters
The simplest filters focus on matching the names of tests and the test suites they are in. The “Test Names Containing” field (-t on the command line) will select all test cases which have the indicated text as a substring of their names. If instead a standard regular expression is used, all tests whose name matches that expression will be selected..
In a similar way, the “Suite Names Containing” field (-ts on the command line) provides a way to select entire test suites based on a similar substring/regular expression search. Note that the string matched is the whole path of the test suite : test suites can contain other test suites.
Sometimes test suites contain different tests depending on the version identifier. In this case, fill in the “Tests for Version” filter to select the tests applicable to a particular version. This is filled automatically if the static GUI is itself started with a version identifier. It is not generally useful to do this on the command line - simply running with a version will have the same effect.
You can also search for certain logged text in the result files. This is done via the “Log files containing” filter (-grep on the command line). By default, this will search in the file identified by the “log_file” config file entry. If the “Log file to search” filter is also provided (-grepfile on the command line), that file will be searched instead. This allows selecting all tests that exercise a certain part of the system's functionality, for example.
If system resource usage testing is enabled for CPU time, you can select tests based on how much CPU time they are expected to consume. This is done via the “Execution Time” filter (-r on the command line). A single number will be interpreted as a maximum CPU time to select. Two comma-separated numbers will be interpreted as a minimum and a maximum. All times are in minutes. In addition, you can use the format mm:ss, rather than needing to convert times into a fraction of a minute, and can also use the operators <,>,<= and >= to specify ranges of times to include.
Reusing such selections : “filter files”
Sometimes it may be useful to define such a subselection of the tests that you may wish to reuse. To do this, select “Save Selection” from the “File” menu, which brings up a file chooser dialog so you can choose a file to save in. Note it has two different options, allowing you to specify that either the exact tests currently selected are to be saved, or the criteria which were used to select them. Whichever, a new “filter file” is created, which can be selected again via “Load Selection” in the same menu, and also via the “Tests listed in file” tab under “Selection”.
The differences between the two variants become apparent when somebody tries to load this file. Loading an explicit list of tests will probably be faster than re-selecting them according to some criteria, but if new tests are added since the selection was saved, it will naturally not pick up these tests.
By default, the static GUI files will be saved in a directory called “filter_files” under the directory where your config file is. The dynamic GUI will save them in a temporary location which is removed when the static GUI is closed. These locations are used to generate the drop-down list for the “Tests listed in file” option, and are also those searched if -f is provided on the command line. These locations can be extended or replaced by defining the config file entry “test_list_files_directory”.
Sorting the test suites
The order of the test suites is primarily defined by the testsuite.<app> files, unless automatic sorting is enabled (see the guide to files and directories) However, there are some quick ways to sort the tests after the fact. By simply clicking on the column header they can be sorted “transiently” (i.e. nothing is saved in any files and the sort is gone if you restart). You can also sort more permanently by selecting the various options from the Edit menu, which also contains various options for manual sorting by moving the selected tests up and down. These options are also avaiable via the “Re-order” submenu in the popup menu for the test window.
Note that by default, sorting a test suite does so recursively (i.e. all contained test suites will also be sorted). To disable this behaviour, set the config file entry “sort_test_suites_recursively” to 0.


Last updated: 05 October 2012