This is why you test results in different markets

It is well known that finance results published in academic papers cannot always be trusted. There is a veritable replication crisis in finance as I have written for example here. Sometimes, a finding can be disproven as easily as just checking the results with international markets as I have done here. And now it looks like the most prestigious finance journal the Journal of Finance may have published an article that doesn’t hold up to even this simplest of tests, an international crosscheck.

Last year, the Journal of Finance published this article that essentially ripped apart the momentum factor. In essence, the article claimed that the stock price momentum factor isn’t a real factor but just a reflection of momentum in other well-known factors. Below is the key chart from that paper. It shows the statistical significance of the stock price momentum factor when one uses a traditional Fama French factor model (yellow bars) and the statistical significance when one adds the momentum in these factors as additional explanatory variables (blue bars0. None of the blue bars shows a statistically significant result, which means that stock price momentum can be explained by momentum effects in factors such as the value or size factor.

Stock price momentum explained by factor momentum?

Source: Ehsani and Linnainmaa (2022)

The practitioner may say: “Who cares?” As long as my momentum strategy outperforms, I don’t care whether this is individual share price momentum or the combination of different factor momentum effects.

To which I say: “There is a big difference between the two results because if stock price momentum is not an independent factor, you can create smart value strategies and other factor strategies that capture stock price momentum without resorting to price momentum.” And since the stock price momentum effect is not only one of the most reliable factors ever discovered but also shows large outperformance it would imply that we can do much, much better with other factor strategies. If the result above is correct, it could usher in a whole new era of value investing, quality investing etc.

Alas, along come Nusret Cakici and colleagues who used the approach above to test the results for the US and 51 other countries. Not only that, but they also played around with the specification of the factor and stock price momentum definitions to test if the results above are robust to changes in methodology. Their results are too extensive to summarise in a single chart or table, so interested readers should check out their paper. But the bottom line of this brute force approach is the following:

The stock price momentum factor cannot be summarised by a combination of factor momentum in other countries outside the US.

Even in the US, the effect found in the study above does not hold once the methodology to calculate factor and stock price momentum is changed slightly.

In essence, it seems that the authors of the study above just got lucky and stumbled over a methodology that got them a positive result that could then be published. And the result is some embarrassment now that things look different abroad than in the US. But for us as practitioners, the key takeaway is not only that we need to test our models in different countries and for different time periods, but also that we can continue to rely on the stock price momentum effect. It remains a strong driver of performance that cannot be replicated by other factors. It’s a unique way to make money.