An interesting way to detect future earnings disappointments
I was reading a paper with the salacious title “Can AI read the minds of corporate executives?” and much to my surprise I learned something wholly unexpected from it. If you want to know what, you’ll have to read the last bits of this post.
As I have mentioned before, AI can assist investors in detecting evasive answers from corporate executives. Executives who provide straight answers to analyst questions and who are able to provide additional detail that goes beyond a boilerplate statement see the share price of their companies boosted.
But if AI can identify evasive answers in earnings calls, think about what AI can do when you train it on the written statements in quarterly and annual filings. It’s not just that we can identify deceptive behaviour and fraud with AI-driven analysis as seen here, but it is also possible to forecast future earnings surprises with such technology.
The problem, however, seems to be that one needs to specifically train sophisticated large language models like GPT4 on regulatory findings. In the study mentioned at the beginning, the AI was trained on the Management Discussion & Analysis (MD&A) section of the regulatory filings of all US companies between 1993 and 2021, as well as the Risk Factor (RF) discussion in the same reports. Note that these sections are not audited (unlike the rest of the report), so management has wide discretion to explain how they see their business and the environment it is operating in. This, of course, provides ample opportunity to obfuscate potential trouble on the horizon and assuage investor concerns.
Using a simple lexicographic approach, where one uses the prevalence of certain words that indicate negative sentiment wasn’t enough to identify companies that in the future had better or worse earnings. Similarly, using off-the-shelf large language models (LLM) worked, but not that well. Standard LLMs are not trained in the interpretation of the specialist language used in finance and ‘corporate speak’.
Once the LLM was trained on this kind of content, it was able to accurately predict which companies would provide positive earnings surprises at the next results and which ones would disappoint. When put into action in a trading strategy, the study found that the 20% of companies with the largest positive earnings surprise outperformed the 20% of companies with the largest negative earnings surprise by 6.7% per year (or 4% per year when adjusted for the usual five factor approach). That is a stunning outperformance if you ask me, and likely to remain in place even if one takes transaction costs into account (which are not insignificant because the portfolio has to be adjusted every quarter).
But while testing their highly sophisticated LLM, the authors also tested very simple approaches to forecasting future earnings surprises, presumably to show how much better LLMs were. One thing they tested was to simply count how long the MD&A section is. And it turns out that the 20% of companies with the longest MD&A section underperformed the 20% of companies with the shortest MD&A section by 2.3% per year (or 1.8% after adjusting for the Fama-French 5 factors). It really is quite simple. If management wants to downplay potential risks, it needs to explain why. And that increases the length of the text. If things are going a.o.k., then all you have to say: Things are going well. Pretty short. So, the next time you read a management discussion of the company, pay attention to how long the discussion is. Chances are that people who write a corporate version of ‘War and Peace’ are trying to hide bad news.
Performance of portfolios sorted on different MD&A attributes
Source: Chapados et al. (2023)