TextTest is about testing at the system level, and it is expected that you are trying to run your system in as similar a way as possible to how it is used in practice. It is, however, more important to have maintainable, repeatable tests that are free of global side effects. Sometimes it is necessary to disable some part of the system, or fake some behaviour that cannot easily be triggered for real in a repeatable way. This is referred to as “mocking out” a subsystem for testing purposes.

(All the following mechanisms involve the test data mechanism : it's probably a good idea to read that document first if you aren't familiar with how it works)

The problem can here be broken down into 4 sub-categories, of which TextTest offers direct help with 3:

When TextTest creates the temporary write directory on running the test, and populates it with test data, it also makes sure to insert that directory at the start of the PATH, PYTHONPATH and CLASSPATH variables. That makes it possible to provide executable programs that will be run instead of their real versions via the normal test data mechanism (“link_test_path”). Likewise, you can provide a Python module or a Java class, which will then be imported instead of the real version at the appropriate moment. Then you just need to place such executable/importable test data in the appropriate place in the permanent test structure, just as you would with more passive test data.

The above mechanism for executable programs is powerful but can be overkill. It's also not very easy to steer in the case where a program is called many times and needs to behave differently on each occasion. It can be better and easier to simply ask TextTest to take over and “record” what the program does. This is known as the command-line traffic interception mechanism.

To enable this, add the name of the program concerned to the config file entry “collect_traffic”. For each test for which you want to mock out this program, go to the Advanced tab under “Running” and check the “(Re-) record command-line traffic” box. TextTest will then create its own fake version of the program and place it in the temporary write directory as if it were test data, as above.

When called, this program will send the command line it was given back to TextTest via a socket. TextTest will then execute it, and record to a new definition file called “traffic.<app>” what the command line given was, what was returned on standard output and standard error and what the exit status was.

Additionally, TextTest will scan the command line for arguments that appear to be files or directories (it will take anything that is an absolute path or a pre-existing relative path) and will store any changes to those files made while the process runs. If the test is saved these will then appear in the test file view under "Externally Edited Files". By default it will only monitor these files while the program in question is running: however you can tell TextTest that it may start background processes that will edit the files, and in this case it will check the files before and after every subsequent received command. To do this, use the "asynchronous" key when adding it to "collect_traffic", which is in fact a dictionary although you don't need to know that unless using this feature.

When the test is next run without the above box checked, this traffic file will be used instead of the real program. As before, the command line is captured and sent to TextTest via a socket. This time, however, it will look up the given command in the traffic file, and for the closest matching command line recorded, will return the standard output, standard error and exit status via the socket, which in turn will be relayed back to your system as if the real program had run. Any file edits that were stored will also be reproduced at this point.

It's then easy to fake certain conditions by simply editing this traffic file by hand, if desired

Sometimes environment variables need to be provided for such programs. As they are run directly from TextTest they don't automatically inherit any environment your program may have set up. In this case you should set the “collect_traffic_environment” dictionary config setting, with the keys being the program names as provided for “collect_traffic” and the values being the names of the environment variables. In this case these environment variables with their values will also be sent to TextTest and will be part of the information recorded.

If your system under test is written in Python you can make use of a variation of this mechanism to intercept and replay the interaction with particular modules in a similar way to that described above. Examples would be things like "smtplib", "xmlrpclib" or "urllib" which may refer to resources that aren't always available where you want to run the tests, or which may cause undesirable global side effects.

To enable this, add the name of the module concerned to the config file entry “collect_traffic_py_module”. For each test for which you want to mock out this program, go to the Advanced tab under “Running” and check the “(Re-) record command-line traffic” box in the same way as above. TextTest will then create its own fake version of the module and place it in the sandbox directory as if it were test data, whilst making sure this directory comes first in the PYTHONPATH environment variable.

By default each such request will be handled by a separate thread so that concurrent calls can be handled correctly (which is also true of the command-line and network traffic interception). Some modules (e.g. Windows COM) however do not appreciate successive calls being made from different threads, and in this case it's necessary to enforce a single-threaded mode for TextTest's "traffic server". This can be done by setting "collect_traffic_use_threads:false" in your config file.

The “traffic interception” mechanism can also be used for this purpose. Here it is less a matter of configuring TextTest and more of configuring your own application. As above, TextTest considers itself to be recording such traffic when the “(Re-) record traffic” switch is enabled, and replaying such traffic when a “traffic.<app>” file already exists. In these circumstances it sets up its own “traffic server” and sets the environment variable TEXTTEST_MIM_SERVER (MIM stands for “Man in the Middle”) to <host:port> for this place.

On testing a client-server system you probably need to write a wrapper script that can for example start server, start client connecing to server, close client, shutdown server. To use this traffic-recording mechanism you should modify this script such that the client will instead connect to the host and port given by TEXTTEST_MIM_SERVER, instead of that given by its own server. You should also modify it so that on discovering the host and port where the real server is running, this information should be sent to TextTest's MIM server in the format <some_message> <host:port>. TextTest will then parse this and know where the server is.

When your client sends a request it will go to TextTest instead. TextTest will record it as a client request in the traffic file, and forward it to the server. The server will then reply, which will be recorded as a server reply, and forwarded back to the client. In this way a complete log of the communication can be built up.

You can then replay this either as client or server. When running the server with a traffic file already in place, TextTest will read the file in order. It will itself send the client traffic in the order it is written down, and each time a server reply is present it will suspend and wait for such a reply. Replaying the client is much the same, except that TextTest suspends initially and waits for contact from the client before doing anything.

This may be easier to understand by example. There is a nice little fake client-server system tested in this way as part of the self-tests, under TestSelf/TrafficInterception/ClientServer/TargetApp.