Investor view: Beware the seductive power of modelling when making investments

Last month, I wrote about the quantification, or McNamara, fallacy, whereby metrics are treated as end goals and a strategy in themselves, and how the advertising industry seemed to have fallen into its own version of the McNamara Fallacy.

Leading on from that, I wanted to highlight another associated issue, namely that while most people focus on the end result from a model, you should be focusing on the inputs.

Models are powerful things. They are often used as the key determinant of decisions. Forecasting is used to drive key decisions across all sectors of business and life, and it is the same with the advertising industry. Facebook and Google in particular have focused heavily in this area and are now moving to replace their old attribution models with new, more econometric-based modelling, which is claimed to be more accurate (ITV has also chimed in with its own model, which has just been launched).

In fact, the “scientific” results provided have probably been the key reason why the online tech giants have consolidated their power over advertising spend and no advertising discussion is complete without mentioning the increasing technicality involved.

Yet, the industry is looking at the wrong thing. As, hopefully, you know by now, for more than 20 years I have analysed companies in the media and tech area, first as an equities analyst in the City and then independently. A key component of my role is building models.

That role has taught me several lessons:

Models are not neutral

The first is that models are not neutral. They reflect the biases – explicit or implicit – of those who build them (it is similar for artificial intelligence).

It is why for publicly listed companies, you see multiple analyst forecasts, even though there is the same publicly provided information. Put simply, if you take the results from a model wholeheartedly, you are also accepting wholeheartedly the biases of those who wrote them.

Models can be manipulated

The second, which is more problematic, is that models can be manipulated to produce whatever result is needed. It is not actually hard to change a model’s inputs to produce widely different results.

It was why I am always sceptical when I see someone using a model’s results to make what they claim is a powerful and persuasive argument.

Unless I know the inputs that have gone into that model and how that model has been structured, it is hard to work out the validity of the claims (without getting into more sensitive arguments, Brexit and climate change are two areas where results from models get used as the heavy artillery in their respective conflicts).

Small changes can make big differences to models

The third is that what also makes models particularly powerful (and make them so suitable for manipulation) is that even subtle changes can cause a disproportional impact on long-term results due to the compound effect.

A classic example is the valuations of tech stocks, where (as with most other companies) analysts typically use a discounted cash flow model to drive the valuation.

Change one or two of the inputs slightly (particularly the Risk Free Rate to base off the discount) and a stock can easily lose upwards of 30% to 40% of its valuation.

The key reason for that is the compound effect, which means that even small changes – particularly early on – produce a “ripple” effect over the long term.

There is a lot more I could say, but space is limited. So what would be my advice on models?

I do have advice for those who build models (essentially based on Abraham Lincoln’s maxim of “Give me six hours to chop down a tree and I will spend the first four sharpening the axe”) but I suspect it is better to advise those who base their decisions on the results of models.

It is this: always remember Ronald Reagan’s dictum “Trust but verify”. It is critical to know what assumptions have been made. Don’t take no for an answer. Once you know the inputs, analyse and question them.

You cannot assume the modellers have produced their numbers without bias nor have "common-sense" checked their assumptions.