Don’t bother forecasting the next 12 months

The title of this post may lead some readers to conclude that in the current environment, there are so many uncertainties that it is impossible to forecast where markets will go in the next 12 months or so. That is true but remember that the world is always full of uncertainties of all kinds and two recent studies show that even if you have hundreds of years of data at your disposal forecasting equity markets for the next 12 months is almost always a useless exercise.

One of the papers that has influenced me the most in the last two decades is “A Comprehensive Look at the Empirical Performance of Equity Premium Prediction” by Amit Goyal and Ivo Welch. They showed in 2004 that the forecasting power of traditional metrics like valuation (P/B-ratio, P/E-ratio, dividend yield) and macro-related measures like the steepness of the yield curve or the Fed model of the difference between earnings yield and bond yield is essentially zero. To quote from their paper: “We find that [a] over the last 30 years, the prediction models have failed both in-sample and out-of-sample; [b] the models are unstable, in that their out-of-sample predictions have performed unexpectedly poorly; [c] the models would not have helped an investor with access only to information available at the time to time the market”.

But that was 15 years ago and since then we have got more data and more research has been done to identify factors that forecast equity markets. Hopefully, with more data and more research we should be able to get better forecasting results. So, the authors went back and re-examined the 17 predictors they tested in 2006 together with 29 predictors that were described in academic journals since 2008. First, they tried to replicate the original finding of these 29 predictors and could do so for 27 of them. Then all they did was to extend the time period of the original data used to describe these predictive factors to 2020. The results were devastating. Of the 29 predictive factors, 25 showed lower in sample predictive power when the data was extended to 2020. Only 4 of the 29 factors had stable or better predictive performance. But that was in sample. Out of sample – which is what matters for investors who want to make money – none of the “predictive factors” managed to beat a simple buy and hold strategy in a meaningful way. One actually did beat it, but only by 0.2% per year. About half of these factors were so bad that they not only underperformed a buy and hold strategy but had negative absolute returns. Investors who followed these factors lost money.

In essence, the new paper of Goyal and Welch shows that the entire factor investing trend is a result of overfitting models to limited data or of false positives and selection bias. It’s not that the researchers who found these factors actively manipulated data or knowingly published factors that don’t work. It’s just that if 20 people look for a predictive factor in equity markets and they all work independently, then by chance, one of these 20 researchers will find a factor that seems statistically significant at the 95% level. And that factor gets published in a peer reviewed journal while the other 19 factors that have failed never see the light of day.

As for the future, I am not hopeful that more data will enable us to get better results. Yang Bai from the University of Missouri created the longest time series of UK and US stock markets possible to create the ultimate backtest. In the UK, he could test macroeconomic and fundamental factors as far back as 1854 and technical indicators like moving averages as far back as 1710. In the United States, he could go back to 1926 for fundamental and macro factors and 1854 for technical indicators. The data he collected allowed him to test 23 “predictive factors” in 312 different setups. Out of these 312 different tests only 12 (3.8%) show an out of sample R2 of 1% or higher and only 19 (6.1%) do so in the United States. In other words, only one in twenty factors can explain more than 1% of the variation in equity returns.

If that depresses you, then let me tell you that next Tuesday, I will write a post that shows that not all is lost. Yang Bai’s study can teach us two important lessons on how to use forecasting models and how to build them.