AI Is Putting Pressure On Software QA. Here’s How Your Team Can Adapt

Welcome to my LinkedIn newsletter! In each issue of Entrepreneurship and Leadership, I'll be sharing my thoughts on scaling, leading and funding high-growth startups and what the future of innovation looks like. Subscribe here to stay updated.

With the advent of AI, a running joke is making the rounds in software circles. Say it traditionally takes two days for a software developer to write code and one day for a quality assurance engineer to test it. Now with AI as a sidekick, it takes just one day to write the code … but two days to test the sloppy results.

That might prompt a laugh among developers, but it’s a serious challenge for software engineers and their companies. AI-driven productivity gains are shifting the balance of labor from writing code — traditionally the lion’s share of development — to everything else that follows before a piece of software goes out the door. 

Where what developers call the Inner Loop has long been the center of the action, now the Outer Loop is growing in importance. I know, it sounds like the plot of a sci-fi movie — complete with highly capable robot assistants.

It’s early days for AI and software development, so this dramatic change hasn’t swept through the ranks just yet. But as the founder of three software firms, I see it coming sooner rather than later. To avoid getting caught flat-footed, companies and teams must start planning now. The Outer Loop will be the pinch point of the fast-approaching future of software development — and unless companies take action, any productivity gains from AI may be, well, a joke.

The Inner vs. the Outer Loop

While the process isn’t always formalized, many software teams apply a similar division of labor when it comes to making their products. 

The so-called Inner Loop encompasses the creative tasks involved in developing software. They include designing, writing, building and debugging code — work performed by individual developers before they share it with fellow team members.

By contrast, the Outer Loop is home to the tasks that include testing the code — doing security, reliability and quality assurance so it’s ready to use. It also covers “artifacts” that signpost the development process, such as data models and documentation. Bookkeeping, admin and meetings also inhabit the Outer Loop.

So how does AI shift the balance between the two? Intelligent agents help developers create code more easily. The mental heavy lifting of translating a concept into strings of code — work that can consume days or even weeks for humans — can be accomplished in seconds by AI. But the results are more like a stream of consciousness than a polished novel. All that raw material needs an editor to prime it for publication.

For example, developers using GitHub’s Copilot AI assistant are 55% more productive. The catch? Copilot yields code with security bugs and design flaws 40% of the time. Combined with the increase in code volume, those vulnerabilities turn the Outer Loop into a bottleneck. 

How will that situation change development teams? Although it varies by organization, a common ratio of developers to testers is three to one. At a big bank with 40,000 software engineers, 10,000 might do security, reliability and quality control. But the AI effect is like squeezing a balloon so it expands on the other side. The coding productivity jump is offset by a dramatic increase in cycles spent on testing.

How development teams can get ahead 

For software teams, the pressure is on to adapt. Companies that want to stay ahead of the game must first get a handle on a long-time adversary: toil. 

Toil refers to the tedious, repetitive tasks that already consume too much of developers’ time and sanity. Whether they’re testing code manually, waiting around for builds to finish or seeking approval to move things along, software engineers find themselves frustrated, slowed down and pulled away from the creative work that drives innovation.  

As AI automation makes the Inner Loop ever faster, the Outer Loop threatens to get bogged down in more toil than ever. The answer: automated, scalable systems for security, reliability and quality.

I’ve seen plenty of engineering teams who are happy enough to hack together their own DIY fixes here. (Ironically, some of the least sophisticated productivity tools in some companies are those deployed by software engineers themselves.) But professional tools are essential to eliminating the toil that still plagues the software development life cycle.

Continuous integration/continuous delivery (CI/CD) is a crucial one. High-performing software teams use it to automate much of the labor needed to push new code through production. Building, testing, deployment — CI/CD takes care of what has historically been a tedious series of manual tasks.

A big part of that is security testing, one of the most time-consuming jobs. Besides giving a clear picture of vulnerabilities, CI/CD can prioritize the most urgent ones and recommend quick, efficient fixes.

Cloud cost management is another Outer Loop task ripe for automation. The right tools can track cloud usage in a high level of detail, shutting down idle resources to cut costs. In my experience, the savings can be as much as 70%.

Even something as simple as internal developer platforms (IDPs) can be game-changing here. These self-service portals help speed up production by gathering together tools, services and information, including an inventory of software components. 

Of course, all of these tools are themselves being enhanced by generative AI, making them even more capable of minimizing toil and accelerating deployment. 

Start developing talent now 

As the Outer Loop expands, it will also fuel a talent shortage that could blindside the unprepared. I’m not kidding when I say we may find ourselves in a world where QA engineers are more sought-after than coders in some circles. Folks who specialize in testing might also start commanding bigger paychecks.

So companies must plan accordingly, by recruiting and building the talent they’ll need to take those changes in stride. Central to that effort: an attractive developer experience. Pay is just a starting point. Developers also need to understand the company’s mission, feel challenged in their work and have the right tools for the job. 

What roles will be most in demand as AI reconfigures the software development life cycle? Expect QA manager as well as engineer to become a hot ticket, along with reliability engineer. 

AI won’t bring this transformation overnight, but software teams that prepare now will be in a better position to get their products to market faster. Ultimately, shoring up the Outer Loop could make the difference between success and failure. That’s no joke.

Thank you for reading! For more insights from my experience as a serial entrepreneur and how we can harness the power of software to change the world, subscribe to Entrepreneurship and Leadership.