This is a basic guide to what the GUI looks like, and how to write your first test. It assumes you have read and followed the instructions in the installation guide.

We will use a 'hello world' program as an example. Text in italics is background information only.

For each executable program to be tested, a 'TextTest application' needs to be created. To do this for the first time, do as follows:

You now have an application with no tests and are ready to run the TextTest GUI. You should now do this, with the command 'texttest.py -a <app>' (if this is your first application you don't need the -a option). You should see something like this. (texttest.py -a hello, after doing the above with a hello world program)

This is the static, or test management GUI. On the left you see a tree view of all tests and test suites that there are. The single line shown is the 'root test suite for the application', created by default, currently containing no tests.

On the right, we have a couple of tabs. The one shown has various options for selecting tests: not very useful when we don't have any. The other “Config” will allow us to view and edit the config file we just created.

To create a test, we first select the root test suite where we wish to add it, by single-clicking that line on the left. This brings up a “Test” tab on the right, which in turn shows us the files contained in the viewed test/suite (currently suite). There is just one, “testsuite.hello”, which defines which tests are contained in the suite. At the bottom are more tabs which are specific to the test we are viewing. We go to the 'Adding Test' tab, enter a test name and a brief description of what the test is for, and click the 'Add Test' button at the bottom. This creates a test and automatically views it:

We now have an 'empty test' under the root test suite (called 'Basic' above). As can be seen, there are no 'standard files' yet, because we have not yet said what the test should produce. We could create these files by hand, but the easiest way is to run the test and see what happens.

We click the “Run” button on the toolbar to run the selected tests. This starts the dynamic GUI in a new window, which performs test runs. The test fails, because no standard results are yet defined. On the left we see a tree view similar to the one above, on the right a summary of which files are different in all tests that have run: useful when we have more! We click on the red test line to see the files it has produced.

It collected the standard error and standard output of the process into 'errors.hello' and 'output.hello' files. These can be viewed by double-clicking in the file view, or a summary is available in the 'Text Info' window. We can see that the text 'Hello world' appeared on standard output, while nothing was written to standard error. We like this behaviour, so we click 'Save' on the toolbar.

Now the test goes green and the produced files are saved as the standard versions. By default, TextTest assumes that succeeded and saved tests are not interesting and hence collapses the suite containing it and removes the Test tab again, returning to the selection view we had before. We can now exit the dynamic GUI, pressing 'Quit'. When we look in the static GUI, which has been there all along, we now see this.

The files produced by the dynamic GUI run have been saved as 'Standard Files' and will now be used as comparison in the next test run (which we would expect to go green immediately).

So there it is: your first working, repeatable test for “hello world”. Now for testing a real program: what do I still need to know?

One of the most immediate challenges is often that real programs write output that changes from run to run, such as temporary file names, process IDs, times etc. TextTest allows you to define what such statements looks like in your program so that it can filter them out, avoiding false failures when only such information changes. To see how to do this, look at the section on run-dependent text.

Another challenge is that real programs often read and write files other than standard error and output. As we saw above, TextTest runs all tests in a temporary directory created specifically for that test run, which is also used as current working directory for that test. You should therefore make sure that your program only creates files relative to this directory, and that all needed input files are available from it. See the guide to TextTest's temporary directory for details.

You may want also want TextTest to compare other created files in the same way as you do for standard output and error. See the section on collating files for details.

At this point you should be able to get going with building a test suite for your application. There is of course more to TextTest than this: look at the reference material available from Documentation Central.