Technology estimates: unpredictable since 1823

In 1823, Charles Babbage was awarded £1,700 to build a machine known as Difference Engine 1, capable of automatically producing astronomical and mathematical tables. It was to be based on Babbage’s success in building Difference Engine 0, a smaller, prototype machine, which showed the potential of automatic calculation.

Nineteen years later, in 1842, after the budget had been spent ten times over, the project was abandoned. By this time, Babbage’s attention had shifted to an even more ambitious project, the Analytical Engine, which anticipated much of the computing architecture of modern, digital machines, including programs, memory and an arithmetic processing unit. Despite the contributions of another genius, Ada Lovelace, only a small part of the Analytical Engine had been built before Babbage died in 1871.

We should not disparage the work of Babbage and his colleagues for being incomplete: the Analytical Engine may be the best example ever of a machine being ahead of its time. To a modern computer engineer, the idea of building a mechanical digital computer out of brass and steel is audacious, to say the least. To a software engineer, the idea of developing computer programs, as Lovelace, without a machine to run them on, without the fundamentals of computing being settled - and without the modern programmer’s primary resources: the Internet, a search engine and online forums - is terrifying.

Furthermore, I believe that Babbage’s work anticipated the modern computing industry in two other ways.

First, it foreshadowed the current wave of enthusiasm for artificial intelligence: incomplete as they were, Babbage’s machines became dinner party sensations, where he would perform calculations for a delighted audience: despite often being characterised as the ur-nerd, Babbage was apparently a charming guest.

Today, we would hesitate to classify Babbage’s machines as AI: they simply performed calculations in response to the turn of a handle. We wouldn’t imagine that something that clicked and clacked could be intelligent. However, we must remember the context of the times. Despite the industrial revolution, machines were confined to physical labour: literal heavy lifting. Mathematical work, of the kind that produced tables, was carried out by humans, using their brains and training. Babbage’s work prompted the question that Alan Turing would attempt to address decades later: can machines think? Today, this reception reminds us that, in a world which often lacks a stable definition of Artificial Intelligence, the old principle that, ‘AI is whatever a machine can do today that it couldn’t do yesterday’, may be useful.

Second, it gave an example of an experience that is unfortunately familiar to sponsors and managers of projects worldwide and through the history of computing: a massive over-run of time and budget, and under-delivery of expected outcomes. However, we should do more than just wryly lament the apparent inevitability of unreliable estimates in the computing industry, and reflect on what it tells us.

I believe that Babbage’s experience should remind us that, when we set out to do computer projects, we are attempting something that no-one has ever done before.

We’ve been building computer systems for decades now. Saying that people building such systems are doing something that no-one has ever done before may seem to be self-aggrandisement, at best, and simply inaccurate at worst.

Yet, if you have experience of delivering at scale and complexity, you have experienced the need to put things together in ways that nobody previously anticipated, including the manufacturers of products and the inventors of computing languages. This may be as small as writing logic which has never been written before, and as large as creating whole new infrastructure components and integration techniques. Indeed, if you’re not doing something new, you have to question why you are doing anything at all: if all these problems have been solved, why not buy the solution from the person who solved them?

I believe that this is one of the many things that technologists are bad at explaining to non-technologists. In our desire to appear professional, knowledgeable and in control, we give the impression that building systems is like building Lego (we even use this analogy): we run our eyes over what’s needed, select the right components from our box of parts, and plug them together in the right way. We hesitate to explain that most of those components don’t fit each other, that many of them have never been used before, and that a lot of them don’t even exist. If we were clearer about the degree of invention that goes into any computing project, then our mutual expectations might be closer to each other.

Babbage was a pioneer, one of the pioneers on whose shoulders everyone in the modern computing industry stands. But everybody who is building solutions does their own little bit of pioneering, pushing further into the unknown potential of computing - and testing our ability to produce reliable estimates.

(Views in this article are my own.)