When TextTest tests are run, it will try to write the output of the system under test to a temporary directory structure created specifically for this purpose. This is known as the “TextTest sandbox”: its purpose is to provide a totally separate environment where your system can create,edit and delete as much as it wants without doing anything permanent, and also an environment where test data can be provided in an easily accessed way.

There is one created for each run of each test. It is created under the directory indicated by the environment variable TEXTTEST_TMP. It defaults to ~/texttesttmp on UNIX systems and the value of the TEMP environment variable on Windows. This is hereafter referred to as the “root temporary directory”.

Each time TextTest is started, it is assigned a unique identifier based on the version and the time stamp at which the run was submitted (the string “static_gui” is prepended in the case of the static GUI). A subdirectory of the root temporary directory is then created with this name. All temporary files and directories created by this run will then be created under this directory.

In the case of the dynamic GUI or the console interface, tests are actually being run. This means that, for every test being run, a temporary directory structure is created which essentially mirrors the permanent directories which represent the tests (see the guide to TextTest test suites), so that each test is assigned a unique temporary directory. All temporary files corresponding to particular tests are then written to these directories. When the tests are run, each test starts the system under test with the corresponding temporary directory as current working directory, with its standard output and standard error redirected to local files.

It is imperative that you ensure any other files created by the system under test are created relative to this temporary directory, to avoid global side effects and to aid TextTest in finding them. This should be possible by always specifying relative paths in your test configuration files, which will be interpreted relative to this directory for each run.

(In the temporary directory for each test case, TextTest creates a subdirectory called “framework_tmp”. It uses this to write its own temporary files, such as filtered versions of the output, performance data etc.)

In the case of the static GUI, the temporary directory will contain logs of each dynamic GUI run that is started from it. These will write files in subdirectories labelled dynamic_run<n>, with the numbers increasing for each run that is started. When the dynamic GUI is closed, the contents of whatever it wrote on standard error will be displayed in a message box by the static GUI, as well as in a file in this directory.

Sometimes the system under test needs to read some file relative to the current working directory. TextTest allows you to place such files in the permanent test directory structure. You should then specify the “link_test_path” config file entry as the (local) file name of the file you want to provide. You can then refer to a local file of the appropriate name in your options file in that test case, for example.

TextTest will look for the file name you specify, first in the permanent directory of the test case, then in the parent test suite and so on up to the root. If it finds such a file (or directory), it will create a symbolic link to it from the temporary directory (UNIX) or copy it (Windows). If it doesn't, it will silently continue, as it is regaded as a normal situation to need test data files for some tests but not others.

The files can be given any name at all (unless the system under test requires a particular name), but beware! For those who have got used to TextTest's naming scheme by now, it can be tempting to add .<app> endings. However, the naming scheme used by files interpreted or compared by TextTest doesn't apply to data files, and such naming isn't generally a good idea as it will convince TextTest that these are files that should be compared.

This search path can also be extended as desired, use the “test_data_searchpath” config file entry. This is a dictionary whose keys are the names given in (for example) “link_test_path” and whose values are directories to look further for test data. The “default” key can be specified to make the same search path apply to all test data.

Sometimes the system under test will itself edit existing files. In this case, you will want to copy to the temporary directory the file or directory structure which it plans to edit, so that test runs are repeatable and do not have global side effects. You can do this using the “copy_test_path” config file entry, which will find files or directories to copy in the same way as link_test_path, and indeed is equivalent to link_test_path on Windows.

On UNIX, there is a third option. Sometimes an application may need to read from a very large directory structure, and potentially edit some files in it. Copying the whole structure for each test run is possible but time consuming. It's better to be able to copy just the parts that will be changed and link the rest. This is done with the “partial_copy_test_path” config file entry, in conjunction with the catalogue creation feature (“create_catalogues” in the config file). The first time the test is run, all the files are copied, and the catalogue records which files are created, edited and deleted. The next time, the structure will be copied and linked as determined by what is in the catalogue file.

If any use is made of symbolic links to the master data, it is generally recommended to make the entire “master copy” of the data readonly, in case bugs in the application would cause it to corrupt the test data. It is possible to tell TextTest to ignore the catalogue file and copy everything again if the file-changing properties of the test change : check the “Ignore catalogue when isolating data” box (-ignorecat on the command line)

Applications will often reference their test data structures via environment variables. When these structures are isolated by TextTest as described above, it can be helpful to update the variables accordingly. There are two ways to do this, which are fairly similar in effect.

The first is to simply refer to the environment variable using one of the test data config file settings described above (link_test_path, copy_test_path or partial_copy_test_path). For example, you could write

copy_test_path:$MY_ENV_VAR

This would take the value of the environment variable MY_ENV_VAR as determined by the environment files and the external environment, identify if it refers to an existing file or directory, and if so, copy that as test data. The environment variable will also be updated to point at the absolute path of the copied location.

Alternatively, you can associate environment variables with test data found via the normal mechanism. This is done via the “test_data_environment” config file setting, which is a dictionary. For example

copy_test_path:data

[test_data_environment]
data:MY_ENV_VAR

For each name identified by link_test_path, copy_test_path or partial_copy_test_path, you can provide an entry which will be the name of an environment variable to set to the isolated version of the data.

Note! In both of these cases the environment variables will be set even if no data is found. The assumption is that the system under test might in that case want to create such data in an equivalent position.

If you specify a directory as test data, via any of the three ways described above, it will be treated as test data recursively in its entirety. Sometimes, however, some parts of it are not really part of the data and should not be displayed as such, either in the GUIs or in the catalogue files. If it is version controlled via CVS it is for example likely to contain CVS directories which we will want to ignore.

To achieve this, use the setting “test_data_ignore”. The keys are names identified by “link_test_path”, “copy_test_path” or “partial_copy_test_path”. The values should be names (or regular expressions) of files or directories under the relevant structure which should be ignored. If a directory is ignored, so are its contents.

This has several effects. These files/directories will not be shown in the static GUI (which will behave as if they didn't exist) and changes in them will not appear in the catalogue file, if there is one. If you are using “partial_copy_:test_path”, that means they will also not be copied, ever: changes there are regarded as uninteresting. It is thus a very bad idea to use this setting in conjunction with partially copied structures, if there is any chance of a write-conflict between several tests, or if files will be created (and not deleted) by test runs in that place. In the last case the directory would grow without limit.

When you quit the GUI (or the console interface terminates), the temporary directory associated with the run is by default automatically removed. It is thus important to save any test results that you wish to use again, either as the default result or as a version.

It is also possible to run TextTest in “keeptmp” mode. This means that the temporary directory structure of the run is not removed when texttest exits. However, beware! In this mode it will clean up the run directories of previous runs that it finds which had the same application and version. This means that you cannot run more than one instance of TextTest simultaneously on the same application with the same version in “keeptmp” mode, or they will delete each other's files.

Running in batch mode automatically selects “keeptmp” mode for the temporary directories. It may also be requested explicitly using the “-keeptmp” option on the command line, or checking the “keep temporary write directories” box in the “side effects” tab from the static GUI.