How Much Testing is Enough?


Posted on: February 8, 2014 by Guy Lidbetter

Testing of IT systems can be an expensive business. Sometimes with all the software tested, you plug it all together and something breaks. It's not just the individual components but how they work together that determines successful operation. Similarly, stuff that works perfectly under a light test load can suddenly misbehave under production conditions. So how much testing is enough?

From mainframe to client/server to object orientation to hundreds of thousands of dedicated IOS and Android apps, there's an old adage that hasn’t changed. Fully testing any application isn’t difficult, it's just a matter of whether you do it before it's live or after. If you ever hear the phrase "what can possibly go wrong?" uttered in the context of deploying an IT system, be very afraid.

As this morning's 1st set of Sochi medals in the snowboarding eluded the hopeful Brits, Atos was embarking on its 7th term as the lead IT Integrator for the IOC and the question of how much testing is enough becomes a little more significant than it might be for the authors of Candy Crush. In such a mission critical environment with any failure in full view of the eyes of the world, the answer is that you test it plenty and you don’t wait for live operation to find out that it wasn't enough.

100,000 testing hours, 2 full scale rehearsals and a risk management based approach to pre-empt the things that could possibly go wrong are all invested into ensuring nothing does. And how long does it all take? 4 years between every games cycle, all to ensure 3 weeks of trouble free operation during which the last thing any of us should be hearing about is the IT.

So next time you're asking yourself "How much testing is enough", you may want to consider something else - "What is the potential price of failure?"

Share this blog article


About Guy Lidbetter

Chief Technology Officer, Infrastructure & Data Management. Atos Fellow and member of the Scientific Community
With over 30 years of experience in the IT services industry, as CTO for Atos Infrastructure & Data Management, Guy is responsible for setting Technical and Innovation Strategy across the IT infrastructure stack in both cloud and non-cloud delivery models. He is also responsible for senior level relationships with technology leaders of strategic partners. Previously, he has held numerous technical and management positions in Sema Group, SchlumbergerSema and Atos Origin. In 2017 Guy was appointed an Atos Fellow and is also a founder member of the Atos Scientific Community, most recently sitting on the Editorial Board for the latest Ascent magazine, “Imagining our Quantum Future’.  He has a passion for sport, particularly Chelsea Football Club, baseball’s Atlanta Braves, rugby union and cricket. He also walks, cycles and more leisurely pursuits include photography, reading, music and attempting cryptic crosswords with varying degrees of success.

Follow or contact Guy