Monday 18 February 2013

In testing prefer lacunae over irreproducibility

When testing you may be tempted to use random inputs. This temptation will occur when you have a large problem space. The wishful thinking here is that the randomness will discover an error where analysis and inspection fail. The tragedy of this approach is that if it were to be successful there is every chance that you could not reproduce the test conditions.

The anxiety which randomness is brought in to quell is that there may be lacunae in non-random generated test data, where two variables vary in lock-step and so never expose a legitimate combination. This anxiety is a genuine one, and is not necessarily exposed by coverage tools. The sticking plaster of randomness is however not the way to address it.

No comments:

Post a Comment