Last week, I have written twice about the poor performance of forecasts in general and in recent years due to the extreme shocks from the pandemic, supply chain disruptions and the Ukraine war. But these are just the most recent posts. Searching through my archive, I have written 51 articles since 2019 with the word “forecast” in it. That is roughly one every three weeks.
The one that I always quote is the one on my 10 rules for forecasting because it is the one that I try to follow in my work and that has helped me through the ups and downs of markets to make decent “forecasts”.
After the extreme shocks of the last two years, many professional forecasters as well as professional investors and geeks who just like to forecast stuff for no professional reason whatsoever are in the process of questioning their models and trying to “build back better” for the post-pandemic world. And here is where the paper by Yang Bai I mentioned last Thursday provides two additional insights beyond my 10 rules.
First, Yang Bai’s study showed that the forecasting power of the technical and fundamental models he tested isn’t always bad. Yes, in the long run, they don’t add value or create outperformance, but when equity risk premia are extremely high, and the economy starts to recover (e.g. after a recession or the pandemic shock of 2021) most forecasting models do work quite well. Similarly, if the equity risk premium is very low and economic growth starts to slow down (i.e. at the end of a boom) these forecasting models again work quite well in indicating future returns.
In other words, when valuations are at extremes and the economy starts to make a U-turn, you better heed the input of these forecasting models. This goes back to an idea I described in chapter 7 of this book on how to forecast a complex dynamic system. In essence you run an entire series of indicators and ignore most of them most of the time. But you keep looking for signs of an indicator being extremely stretched. The indicator that is stretched the most and at the most extreme levels will influence the market next.
For example, throughout 2021, the P/E-ratio of the S&P 500 was hovering around 24.5x trailing earnings. Expensive, compared to the 40-year average of 19x and the most expensive valuation since 2001, but did these extreme valuations stop the US market from rallying? Of course not. The total return of the S&P 500 in 2021 was 28.5%. What did stop the recovery (at least for now) was inflation that rose to the highest level in 40 years and a Fed that embarked on its most aggressive hiking schedule in at least that long. Inflation today is truly extreme, so investors pay attention to it and the market reacts. Valuations are extreme but not extremely extreme (if that makes sense), so markets ignore them until they no longer can.
Second, Yang Bai showed that if you build a forecasting model based on UK data and then apply this model to the US market these models tend to work quite well in both countries. Meanwhile, if you build a model with US data and then apply this model to the UK market these models tend to perform poorly in the UK but good enough in the US. Why is that?
I think one reason for this is that the UK market simply had more extreme events in the past than the US market. Think about it. Throughout the 19th and 20th century there were hardly any major catastrophes that directly impacted the United States. The Civil War and the Great Depression come to mind, but the United States were relatively unscathed by the two World Wars or experienced extreme hyperinflation or a sovereign default in the last 150 years.
Compare that to the experience of the UK, Germany, France, and Russia. One paper that professional investors are not very fond of quoting is “Global stock markets in the twentieth century” by Philippe Jorion and William Goetzmann. Their paper shows that between 1921 and 1996 the real return of most equity markets (i.e. equity market return above inflation) is not statistically significantly different from zero. Equity markets with a statistically significant return above inflation are the United States, Germany and Japan since the end of the Second World War, Sweden, Switzerland, and the UK. But these are all markets that didn’t experience any really extreme events like wars on their home soil, hyperinflation, sovereign default, etc.
But if you calibrate a model on benign historical data, you are running the risk of overfitting a model that will eventually just produce noise and unreliable forecasts, particularly if the underlying driver of returns change. Just think of all the models that predicted mortgage defaults before the 2008 financial crisis and assumed that house prices in the United States never drop everywhere at the same time.
If you test a forecasting model with US data because it is easy to get long time series and after all, the US is still the world’s largest market, then you are constantly committing the same error as these mortgage modelers before the financial crisis. And while your models may perform ok for a while, they will eventually fail and become costly errors. Better to develop a model with data that is “unusual” and has more extreme events or regime breaks in it. These models are less at risk of being overly optimised to a benign world and more robust to sudden shocks and regime shifts. And the one thing I think we can be sure of going forward is that there will be more sudden shock that nobody expected.
reverse stress testing