Is software quality possible?

Why is quality in Computer Engineering always a feature to be sacrificed when we are short of time/money? It is curious how, in other branches of Engineering, nobody would question the importance of quality. There is a sentence that a professor of Quality, Safety, and Auditing told us during a lesson: “Would you pass over a bridge made by a friend who has studied Civil Engineering?”.

The answer, unless we don’t like (or like very much) our friend, is undoubtedly yes. However, when we change the question to “Would you get on a plane whose software was developed by a friend who has studied Computer Engineering?”, the answer is simple; if you have also studied Computer Science, 10 times out of 10 you would answer no. This is simply an illustration of what quality in software design and implementation means, or in other words, the lack of it.

“Would you get on a plane whose software was developed by a friend who has studied Computer Engineering?”

When leading projects, especially if you don’t have accumulated experience with the quality processes to be followed to ensure successful delivery, many mistakes are made that later cost a lot of money, time, and effort to correct. When you do not have a quality standard to follow, whatever it is, the quality standard then becomes the experience of the person leading the project.

But experience has a disadvantage, which is not transmitted from person to person simply by studying it; it is necessary to fail, and fail many times where others have already failed, in order to achieve it, something that in my opinion will generate more waste of money, time and resources than having a quality plan developed and ready to be implemented, however simple it may be.

A study showed that a programmer committed, on average, 100 defects for every KLOC (1000 LOC) lines of code introduced

These three examples illustrate what has been discussed above:

1. A new commercial “self-service” subscription system via the web was implemented on the Paris underground. A user entered by mistake a different URL in the browser and thousands of records of confidential and personal data were deleted. The URL executed a deletion policy that was not previously validated.
2. In Japan, an operator of the Mizuho company mistakenly typed a sales order of 610,000 shares at 1 yen, instead of a sales order of 1 share at 610,000 yens, into the software used for share exchange. The “cancel” functionality was not operational, resulting in losses of USD 225 million.
3. Playstation 3 suffers a worldwide connection error due to the change of month. The error “8001050F” occurred when the console tried to jump from the last day of February to the first of March.

Application crashes, data corruption, performance, and scalability or security problems are just some of the nightmares of every software company because, in addition to the logical consequences that all this entails, there are other related hints, such as layoffs, bad image, customer insecurity, etc. However, errors are inevitable.

To give you an idea, a study showed that a programmer committed, on average, 100 defects for every KLOC (1000 LOC) lines of code introduced. In addition, the success of testing (error detection) is less than 50% and normally, as we mentioned at the beginning, testing (or certain levels of it) is dispensed with throughout the development of a project.

 It is interesting to note that IBM conducted an experiment where its programmers were put under extra pressure for a controlled period of time and found that defects introduced in the code were multiplied by 5 during this period of time. If we add to this cocktail that in the applications, as they gain maturity and functionality, the number of use cases to be tested is multiplied over time and the probability of covering them is reduced, it seems impossible that only with testing we will be able to deliver quality software.

So, is quality not achieved by improving the tests? The answer is no because after the tests you will not get quality unless you get something that already has quality. So, what can be a solution? We must work with quality from the beginning of the project.

Of course, improving the tests is an important complement, but it is only one piece of the puzzle. And we must break the myth of the inverse proportionality between quality and productivity: “If you increase quality, you reduce productivity”. These aspects, which should be taken as complementary, are generally taken as antagonistic.

This graph illustrates the cost reduction and productivity increase in 1994 of Raytheon, the world’s first missile manufacturer in the USA. We can see how before 1988 the percentage of ROI reinvested in correcting errors introduced in the code could even approach 50% in some cases. From 1988 onwards, quality processes were introduced, and it can be seen how the percentage of money spent on fixing errors decreased drastically compared to the increase in productivity of its developers. Obviously, and as we mentioned before, it will never be zero since we have to be aware that errors occur and will always exist, but we must mitigate them as much as possible.

There are models that define how quality processes should be, models that are difficult to apply at the beginning, and which are made up of different levels of maturity that each company can progressively adapt. I am referring to the ISO 9001 or the CMMI model. However, this article is more focused on those companies that do not follow a defined model but that also want to incorporate quality into their processes without losing sight of their goal.

“If you increase quality, you reduce productivity”. These aspects, which should be taken as complementary, are generally taken as antagonistic.

To summarize, I would like to stress that we cannot focus on just one part of the process, the testing, but must focus on increasing the control framework throughout all the stages that make up the project. Let’s not forget that we are in an agile world and that each iteration should be short and capable of adding value, allowing us to go back without creating much chaos and letting the product evolve while we get feedback from the market. As we move away from the waterfall world, we must be clear about which quality model has been established in our company and be able to follow it and evolve it according to the experience gathered and, above all, write it down.

It has to be accessible to everyone, governed by a group of people directly linked to the management and exclusively dependent on it (quality must not be subjective), and that it serves as a reference for all members of the organization. And finally, it is not necessary to follow an ISO 9001 or CMMI model; quality can start with oneself.