How Much Testing is Enough?

Testing of IT systems can be an expensive business. Sometimes with all the software tested, you plug it all together and something breaks. It's not just the individual components but how they work together that determines successful operation. Similarly, stuff that works perfectly under a light test load can suddenly misbehave under production conditions. So how much testing is enough?

From mainframe to client/server to object orientation to hundreds of thousands of dedicated IOS and Android apps, there's an old adage that hasn’t changed. Fully testing any application isn’t difficult, it's just a matter of whether you do it before it's live or after. If you ever hear the phrase "what can possibly go wrong?" uttered in the context of deploying an IT system, be very afraid.

As this morning's 1st set of Sochi medals in the snowboarding eluded the hopeful Brits, Atos was embarking on its 7th term as the lead IT Integrator for the IOC and the question of how much testing is enough becomes a little more significant than it might be for the authors of Candy Crush. In such a mission critical environment with any failure in full view of the eyes of the world, the answer is that you test it plenty and you don’t wait for live operation to find out that it wasn't enough.

100,000 testing hours, 2 full scale rehearsals and a risk management based approach to pre-empt the things that could possibly go wrong are all invested into ensuring nothing does. And how long does it all take? 4 years between every games cycle, all to ensure 3 weeks of trouble free operation during which the last thing any of us should be hearing about is the IT.

So next time you're asking yourself "How much testing is enough", you may want to consider something else - "What is the potential price of failure?"