The Leadership Guide to Holistic Testing and Devops

Welcome to our first industry-focused newsletter. This newsletter is inspired by our recent visit to Edinburgh and the talk by Janet Gregory hosted by the British Computer Society. The talk addressed DevOps and implementing Holistic Quality within DevOps delivery. So many things that were said rang so true and echoed our experience. We thought that we would share our thoughts.

The Holistic Quality approach resonated with us. Quality reviews of each artefact are common to many Engineering projects, and consideration of testing at the earliest point is also a practice that we strongly believe in. We liked the DevOps diagram augmented with each stage annotated with “Test Here”. Janet made a good point in that it may be more appropriate to use check rather than test, as there are many ways to improve product quality; these include testing and other evaluation techniques such as reviews.

The discussion on quality levels was particularly good. Janet articulated that the quality achieved will vary between organisations and projects and that the highest possible level of quality is not necessarily optimum for each project. Anyone would expect the quality requirements to be lower for Proof of Concept software than for safety-critical flight software. The important aspect is for the team to decide on and implement the level of quality that applies to them.

A further aspect that resonated with us was that of Eliminating Bottlenecks. It has been our company theme for several years. Testing is sometimes a bottleneck, particularly when an end-to-end integration test is used to validate that the system meets business needs. We appreciated the discussion on the test strategy and ways to test to avoid failures at the final hurdle. These include good, modular system design with well-defined and readily tested interfaces and using mocks to provide functional confidence before the end-to-end test. We discussed instances in which the full end-to-end tests were mandatory and whether automating the tests in these cases was sensible.

We thanked Janet personally following the session. We thanked her, in particular, for her thoughts on test tool selection. Different test environments present challenges; for example, installing test automation software on some systems is prohibited. The advice we respect is to address each test automation challenge individually and select the tool most appropriate to the technology and the skills within the team.

Visiting Edinburgh and meeting one of our professional heroes in person was fantastic. Her wisdom and Knowledge have reinforced our dedication to staying at the forefront of our field and continuously raising the bar!