How to Save Your Budget with Timely Qualityby COO Victoria Moiseyenko on November 11, 2016
You’ve probably heard about an American $1 billion rocket which destroyed itself 40 seconds after it took off because of a software bug. You probably think that only bugs on such large-scale projects can cause such gargantuan damage.
Unless you are going to launch a shuttle there is nothing to worry about – some might say. If any bugs arise, they can simply be fixed in a testing or a post-release phase, right?
However, to understand real implication of bugs on different development stages and importance of fixing them as early as possible and preferably preventing them, let’s take a look at the bigger picture.
As an IBM white paper says, the earlier you fix, the less you pay.
The expenditures for fixing a bug in the testing phase is about 10 times more than if you detect and remove it at the requirements phase. It costs up to 30 times more to fix the same bug when the product has already gone live.
Consider the whole development process as a system of interconnected stages.
The higher level the problem goes to unattended, the higher risks are that the problem won’t be fully solved or its solution would trigger other defects, and that means – more money, time, people, and so on. An endless loop.
To benefit from testing to maximum, you should understand that the more accurate and meticulous testing is done at the previous phase (starting from requirements and design phase), the more effective testing and efficient usage of resources will be possible at the following stages, leading to post-release phase with a miniscule number of bugs.
Testing and fixing bugs which start ONLY at the testing stage
- won’t ensure maximum quality of an app,
- take up too much additional time and too many additional people,
- means spending lots of money which you can use in a more businesslike manner.
Let’s take a closer look at astonishingly different time and costs needed to fix bugs at each stage of development.
1. Requirements and design. Test your map before setting off!
The cost of fixing bugs on the requirements and design level is based on additional tester hours.
Think of requirements as of a map. If a map has inaccuracies and mistakes, the ultimate desired destination may be never reached.
The amount of bugs introduced in the requirements and design is the largest – about 45% of the overall amount of bugs found, and they are the hardest and the most expensive to remove, when found on the production level.
Methods to remove them at the early stage:
- A proper process of analyzing and interpreting the requirements to properly transfer the customer needs into specifications.
- Requirements testing. Requirements should be clear, feasible, measurable (with specific values), testable, with no contradictions. It should be checked whether no requirements are missing.
- TDD is one of the best ways to make developers get focused on the requirements, before getting down to coding. It’s like emulating the whole way till the destination is reached using the guidelines (requirements).
2. Coding. Let programmers check their code.
The cost of finding and fixing defects on this level is based on additional developer hours.
- Unit testing. Unit tests are written by programmers to check whether a certain piece of code (a function) is working as it has to. Read on importance of writing testable code!
- Code review (self-review, peer-review ).
Code review is aimed at understanding whether the code is doing what it is supposed to. It is an extremely useful process since it helps to get rid of the algorithmic issues, missing conditions, etc.
3. Testing. Use it as a foundation for further analysis.
The costs on this level are based on a tester, developer, system engineer, PM hours.
Testing level involves more people and entails much more activities than the earlier stages: bug detecting, logging reproduction steps, prioritizing a bug, meeting and discussing with responsible programmers, fixing a bug, deploying the fix to the test environment, verifying a fix, regression testing to check whether a fix has injected new bugs into the system.
Proper defect logging and analysis of bugs is mandatory for successful bug fixing, analysis, systematizing of bugs, and developing effective preventive measures for the next builds.
Proper defect logging:
- Describing a defect in a way that everyone on a team understands it correctly, with accurate and succinct steps to reproduce it;
- providing screenshots for illustration and better understanding;
- indicating the name of a person who found a defect for improving traceability and knowing who to address if a need arises;
- indicating a phase where a bug was found.
The defects should be recorded in a way to create a solid foundation for analysis and developing a systematic approach to taking effective preventive measures.
4. Production. Don’t let your money down the drain!
Fixing bugs on this level is the most expensive one.
The costs are based on a support desk worker, system engineer, QA, PM, developer, and customer hours. And don’t forget about financial losses a business incurs if showstopper bugs arise in production.
If a problem occurs in this phase, support workers are contacted, and it’s up to them to decide whether it’s a defect or a part of functionality. The customer is informed. The PM is contacted and the testing team is involved. The activities from the testing phases are undertaken, including regression testing.
Due to urgency of emerging issues, the process calls for more rigorous planning and more prioritizing than in the previous phase.
It needs to be done as soon as possible since it’s already in production. However, it takes time, and business incurs considerable losses.
Though having minor bugs on this stage is almost inevitable, effective testing on the previous levels can ensure that no critical defects are spotted by app users.
The takeaway. Develop a systematic approach!
Root Cause Analysis
The most important takeaway is to collect, analyze, systematize all the logged defects in order to grasp tendencies and patterns, identify the root causes, and build working strategies to prevent the most frequently occurring bugs in the following builds starting from the very beginning (the requirements phase).
Identifying, analyzing, and preventing systematic errors is key to the most effective and efficient usage of resources: time, money, and people.