Fooled by randomness

The excellent Past, Present, Future podcast recently had a series of episodes on historical counterfactuals. I was looking forward to the series as a stimulating intellectual exercise, but I admit I was quite disappointed. This got me thinking and at least in that respect, the podcast was intellectually stimulating. I think in social sciences we are treating randomness all wrong.

If, like me, you trained in natural sciences, engineering, or medicine, you will know that models tend to focus on the driving force that explains nature and are usually accompanied by the famous e, a random error term.

In their physics envy, economists also create models that contain a description of the driving forces of GPD growth, inflation, stock markets, etc., and then add a random error term to capture the fluctuation and noise encountered in real-life markets.

However, in my view, there is a category error in the models used in economics and finance and those in science and engineering. In physics, chemistry, etc., the random error term really is an error term because we can observe a chemical reaction or run lab experiments thousands if not millions of times. This allows us to amplify the signal and reduce the noise to understand the laws of nature and how the world works.

But in economics, finance, politics, and other social sciences we cannot do that. We can only observe one realisation of an inherently random process. There is no way to run the year 2024 thousands of times to see which economic or market development is just noise, and which one matters.

Yes, we can try to separate the signal from the noise by looking at many years of historical data or many thousands of stocks hoping to find the common driver and reduce the random noise, but this is where I think we are making a crucial mistake. The world changes all the time and there is no way to capture all the ways how the world changes and how different assets react to these changing drivers. Hence, unlike in science and engineering, the error terms never really cancel each other out and the signal-to-noise ratio cannot be improved to a level where you can truly ignore the noise.

To give you an idea of this effect look at the forecasting error terms for future equity returns shown in this famous paper by Pastor and Stambaugh. It shows the error in equity return forecasts for increasing investment horizons in the top panel and the different components contributing to this error in the bottom panel.

Forecasting error for stock markets

Source: Pastor and Stambaugh (2012).

As you can see, the error term does not decline over time but instead increases. Looking at the bottom panel shows that mean reversion leads to a reduction in error over time. Meanwhile, the iid term (the famous e in models) remains constant over time. If we had only those two error components the forecast error would indeed drop if we forecast long-term returns vs. short-term returns.

But we also have an error from an uncertain future and a growing error from compounding the initial error over time as well as rising forecasting uncertainty. These errors grow more and more as we go further into the future and overwhelm the iid and mean reversion effects. No matter what we do in economics, finance, and most social sciences like politics, we cannot reduce these errors.

This is why these episodes about historical counterfactuals were so disappointing. They tried to make a case that for example the French Revolution or the Industrial Revolution could have happened in Asia instead of Europe because of some similarities in the economic and social structure in China or Japan that were similar to France or the UK at the time.

But in my view the underlying assumption, namely that certain economic or political developments necessarily triggered the French Revolution or the Industrial Revolution, is wrong or at least doubtful. The French Revolution may have been partly triggered by social inequalities and the industrial revolution was partly triggered by the invention of the steam engine, but there is nothing deterministic about these events. The French Revolution may as well not have happened at all, and the steam engine may as well have been invented in France rather than England and world history would have been completely different.

But – and this is important – we cannot know the true answer because we cannot re-run the experiment. We cannot re-run history. We always and everywhere have just one instance that we have to interpret.

And interpret, we do. We try to come up with counterfactuals to see what could have happened if the French Revolution had taken place in China or the steam engine had been invented in France. But this will never reduce the error we make in interpreting history because we are trying to create an alternative history starting with a single assumption. And that assumption is guaranteed to be at least partially wrong.

Starting with this flawed assumption we then make more assumptions and more errors, and these errors do NOT cancel each other out. Just like the forecast errors for stocks do not cancel each other out over time. Instead, we are getting further and further away from the truth and creating a complete fiction that we then try to interpret to find meaning in the one instance of history we have observed.

It is the reason why futurists are so useless. They are compounding errors to come up with a prediction that has pretty much zero chance of being anywhere near the real outcome.

Similarly, in politics, people are trying to predict elections, wars, etc. but the best we can do is come up with some likelihoods. Even so, these probabilities can never be verified or falsified because we can run the US Presidential election only once. We do not know how the US would have fared if Donald Trump had lost against Hilary Clinton in 2016 or won against Joe Biden in 2020 and there is no way to reduce this uncertainty.

We cannot know if central banks could have avoided the inflation spike and recession that followed in many countries in 2022 and 2023. We cannot re-run history and we cannot assess if the many prescriptions critics of central banks had on how to better manage inflation would have worked.

The problem is with the error term. In economics, finance, politics, and most social sciences we create models with an error term and then conveniently ignore the error term as ‘noise’. But nobody is ever taught how to interpret noise even though the noise part is arguably at least as important as the assumed signal.

People are inherently reluctant to accept an outcome as random and always look for patterns to explain events. This is what leads to grand theories of history as well as conspiracy theories. I strongly believe that the most underrated law in society is Occam’s Razor which states that the most likely explanation is the one that makes the fewest assumptions. I strongly believe that in the vast majority of events in society, history, economics, or markets there is one explanation that makes the fewest assumptions and that is almost always correct: things happen by chance and there is no explanation for it other than ‘stuff happens’.

We are trained in school and at universities to follow our instincts and ‘explain’ these events by looking at common factors that drive different episodes and ignoring the noise. Instead, what we need is an entirely new field of research that is also taught in schools and universities: How to identify and deal with random events when the noise cannot be reduced. You may want to call it ‘improvisational science’.

All sports commentary, economics commentary, politics punditry, etc.

Source: XKCD