You have probably asked yourself this question more than once. So have we. Really satisfying answers are unfortunately rare. Sure, I have test design techniques that suggest I have tested everything important — but it is easy to miss the actual topic. Sure, I can ask the stakeholders. The answers tend to be things like "just test everything." Not very helpful. Sure, if I ask developers, unit tests alone will suffice. That rarely aligns with the sense of quality. And if we are honest, two questions are being conflated here: are we testing the right things, and are we testing them correctly? So the real question has to be: are we testing the right things correctly?
Methods and techniques help here, no question. Yet in testing, there are always blind spots in our strategy and approach. Classically, maturity models address this topic — but they tend to cover topics and aspects of the standardisation school of testing. TMMI and TPI are the standard bearers here. In the agile world, we lean more on approaches like continuous improvement through retrospectives. We look at what works well and what does not. Blind spots, however, are not to be underestimated.
Callum Akehurst-Ryan gave a talk at Agile Testing Days on a concept called Quality Radar, which originates from Nicola Sedgwick. Similar to maturity models, it identifies possible testing activity areas that can be questioned for coverage all at once. If you ask us, this sounds intriguing and is worth a look — because this concept takes into account a few aspects that are otherwise easily overlooked. Not only internal quality aspects such as unit testing and external quality aspects such as requirements-based testing are considered, but also brand quality topics such as user surveys or Google Trends. This is complemented by 4 dimensions of analysis. Anyone reminded of Brian Marick's test quadrants model is not wrong — it is explicitly named as an inspiration, but is refined and reinterpreted. The Quality Radar allows blind spots to be identified across very different quality perspectives — technical, functional, user-centred, or business-strategic. While classic maturity models recommend a clear process path, the Quality Radar invites a more interdisciplinary engagement with quality. And as always, interdisciplinary compositions are beneficial to quality, because they allow us to combine different contexts and perspectives.
Curious whether you are testing the right things? Perhaps this article will serve as inspiration: https://medium.com/cazoo/quality-radar-a-new-way-to-visualise-quality-f1131668cf95
And for even more inspiration, take a look at our testing toolkit: https://www.oose.de/seminarbaukasten-testen
