On the surface, the question of which environment to conduct software testing appears simple: in test or production? But that seemingly simple choice of whether or not to test exclusively in production (TiP) can have disastrous consequences.
Software testing is unquestionably a necessary part of the software development lifecycle (SDLC). It provides developers the chance to ensure that their product performs as planned, meeting both business and user expectations. In other words, it’s the last chance before the product is available to users to weed out hidden bugs or other glitches.
So why does it matter where the testing occurs? Typically, testing phases (including performance, load, and stress testing) stretch every aspect of the software to its limits in a secure test environment to discover what it can handle, how the users will experience it, and any bugs hidden in the code. The problem? These test conditions don’t always accurately represent the production (a live and user accessible) environment.
Yet conducting tests in a production environment presents some issues, namely that users may interact with the software at the same time you are testing it. If they find a glitch before it is repaired, you run the risk of at minimum providing a poor customer experience. But when today’s users quickly abandon a malfunctioning application, this could spell doom for your project. At the worst, only conducting testing in production opens the door to inaccurate data, slow user interface response, glaring weaknesses, even a destroyed reputation.
Even when faced with hard deadlines to get the software pushed to production, play it safe. Testing beforehand ensures that every wrinkle is ironed out before the user even logs into the software, guaranteeing a cohesive product and a positive customer experience.
Instead of considering TiP as your sole testing method, use it as a spot check. Are you receiving multiple reports of users struggling to complete checkout on the website? Or mobile app software taking too long to respond to requests? TiP is ideal to quickly jump in the production environment to examine and evaluate the situation.
Don’t throw away your last chance to safely review and examine the software. Test beforehand. It isn’t worth the risk.