The test writer doesn't have to think about all the things that
need to be asserted because the logs generated by the system do
it for them. They can concentrate on setting up the test in the
way they think is appropriate, and reading the logs to check
that they provide an accurate summary of what happened.
It is also trivial for a
non-technical person to express wishes for different behaviour
than the current one by simply adding free text to a log file.
In contrast, learning how to call an API and write test
assertions is pretty like programming, however simple the
particularly important when adding testing to a legacy system
'after the fact'. Log statements can be added to it in an
exploratory manner without any risk of breaking it. In
contrast, refactoring a poorly-understood system in order to be
able to create an API so that tests can be written is fraught
with difficulties. Entire books have been written on how to
minimise risk during this process – but why undertake it
at all? Even if the system is well understood the process is
still time-consuming and requires “refactoring without a
has a few implications. It makes writing tests faster. Code
needs to be maintained and can contain bugs – potentially
bugs that hide problems in your real code. No code also means
no dependence on the internal design, so a system refactoring
can never require changing the tests, turning them into a solid
rock to lean on while you do refactoring. This in turn
encourages more ruthlessness in refactoring, by removing fear
of having to rework and potentially break lots of tests..
The creation of an API to provide assertions means a new way of
running the system has been created – which can be
different from running the system for real in subtle ways.
Differences here can naturally result in uncaught bugs. Also,
the more the test system and real system diverge, the harder it
is for customer representatives to interact with and gain
confidence from the test system.
assert things by hand, you will only ever catch bugs that were
exactly what the test writer had in mind at the time. But every
line of every log file in every test is an assertion
that the system is behaving in a particular way. If it ceases
to behave in that way, you want to know about it – never
mind if the test wasn't written with exactly the current
problem in mind. If the change is correct and expected, you can
express that with one click and save the new result.