Testing GUIs with TextTest and StoryText
Developing StoryText
Overview
StoryText is a work in progress and it doesn't support everything you could ever do in the various GUI toolkits. It supports what its users have needed so far, and there is a complete list of what is currently supported here for PyGTK, Tkinter, wxPython, SWT/Eclipse RCP and Swing respectively. You are therefore reasonably likely to find that some widget in your application isn't supported, and you are of course encouraged to submit any changes you make back here.
Checking out the latest version of the code
The code is hosted in Bazaar on Launchpad. Some instructions for checking it out are on the download page.
Running the self-tests
Check out the self-tests via "bzr branch lp:/~geoff.bache/storytext/selftest-trunk".
The self-tests expect you to set TEXTTEST_HOME to the parent directory of your branch, they can then live alongside other tests you write in a single directory structure if you want to set it up like that.
On UNIX, you probably want to make sure you have Xvfb installed, otherwise you'll get lots of GUIs popping up on your screen.
The tests assume that the source to be tested can be found in the location $TEXTTEST_HOME/../StoryText. If you put them somewhere else you'll need to tell it this. This is controlled by the setting in the file default_site/site_configfile in the tests: you can either edit this locally or preferably, copy the file to site/site_configfile and edit it there, where it will be read instead of the default_site location. This also means it won't show up as a change when you check status in Bazaar. You can also make other locally-relevant configuration changes for the tests in that file.
Note that StoryText is only runnable directly from its source tree when used on Python GUIs on Linux now. On Windows, or with Java GUIs, it will need to be installed using a command like 'python setup.py install' or 'jython setup.py install' respectively. Doing this after every edit is something of a pain, so we suggest making use of virtualenv. You can then create yourself a new clean installation and install a link to your source location there, and edits will automatically be used. For example, to set up a jython development environment on Linux, you might do
$ jython virtualenv.py ~/.local/jython
$ export PATH=~/.local/jython/bin:$PATH
$ pip install -e <path_to_storytext_source>
You can then set your tests to run the location ~/.local/jython, and you won't need to run the install command all the time.
We make every effort to ensure that the self-tests are always green. If they fail on the current source for you, the first thing to do is to check their status in the nightjob. This page is updated nightly with the result of test runs on Linux and Windows using different GTK versions. If something is failing you can check in the nightjob history to see if it is also failing there.
Editing the documentation
The source for this website can also be retrieved from "lp:~geoff.bache/texttest/website-trunk" if you find errors in the documentation, or your changes require extra documentation.
Very high-level StoryText design overview
Conceptually, StoryText consists of a Replayer, a Recorder and a Describer. An effort has been made to keep these distinct from each other and not excessively interdependent. Organisationally, it consists of generic code that does not know about GUIs, (most of the base package), code that depends on GUIs but not on any particular toolkit (the guishared module) and then the various toolkit specific code, in the respective subpackages. StoryText is not normally used on non-GUI programs, but as it handles signals on POSIX and application events (waiting), it is useful to keep these parts separate, not least to be able to test them without involving GUIs.
The Replayer and Recorder work with "events", which are instances of subclasses of definitions.UserEvent. This concept encapsulates a kind of event that can occur, and each "toolkit" package will generally have a simulator module that defines a number of subclasses of this. Each of these define a generate method with the code to simulate (replay) this event. Often this will forward to a third party tool (RobotFramework SwingLibrary for Swing, SWTBot for Eclipse, i.e. SWT, RCP or GEF). They also define a connectRecord method, called to attach a listener as appropriate to listen for the event occurring. In this way the nuts and bolts of recording and replaying are usually defined in these UserEvent subclasses, while the replayer.Replayer and recorder.Recorder define the basic framework of how this works in general.
The ScriptEngine class is something that the various toolkits override. It provides a static mapping eventTypes, which maps each widget type in the GUI toolkit to the corresponding UserEvents. It also provides the ability to override how the system under test is started, and to hook in basic documentation.
The Replayer operates in different ways in different toolkits. For the Python GUI support (PyGTK, Tkinter and wxPython) the replayer does its work in idle handlers rather than in a separate thread. A separate thread is used for Java Swing and for plain SWT. The Eclipse RCP support has a moderately complicated callback mechanism, including some Java code so that the replayer can be started in a separate thread within the Eclipse structures and triggered from within Java. A difficulty with this code has been the ease with which classloader problems occur, as Eclipse and Jython do not seem to play well together in this area.
The Describer generally works by traversing the widget hierarchy, checking the type of widgets it finds against the ones it knows about, and building an appropriate method name (a widget of type "Button" will cause getButtonDescription to be called). Most of the toolkits disable the describer when recording. When replaying, the widgets that have state are stored in a mapping widgetsWithState and these are checked after each action is replayed, with changes being described if anything has changed. Only the older PyGTK tries to handle the Describer via listeners, and to enable it also when recording.
Customization involves creating a file called "customwidgetevents.py" and placing it somewhere on PYTHONPATH or JYTHONPATH as appropriate. This will then be imported, and any UserEvent subclasses defined in a module level variable called customEventTypes will be added in addition to those defined in ScriptEngine.eventTypes, above. More detail about this mechanism can be found on the page about custom widgets.


Last updated: 08 July 2015