Using AI in an earnings call

Yesterday, I wrote about how machine learning models are biased similar to humans. But while AI and machine learning methods may be biased, they can still help humans in their investment decisions. One interesting application has been documented in earnings calls.

I have written before about how earnings calls and annual reports can be manipulated to deceive investors. Back then, I also pointed out that new AI-driven products like Deception and Truth Analysis (D.A.T.A.) can help investors discover incidences of deception and obfuscation.

A group of researchers has employed large language models like chatGPT to discover boilerplate answers by executives on earnings calls and analysed what that means for investors. The way they did this is to ask chatGPT to answer the questions analysts pose on an earnings call and then compared the answers of the AI to the answers given by the executives.

Obviously, large language models know nothing about a company except for what has been published in regulatory reports like 10-Ks or annual reports. So inevitably, if you ask them something about a company they will provide more or less boilerplate answers that regurgitate what is already known.

By comparing the actual answer of an executive to the same analyst question with the answer generated by a large language model the researchers could create a measure they call Human AI Difference (HAID). Effectively, a higher HAID indicates that executives provide additional insights to analysts that have not been disclosed in official company filings. A lower HAID, on the other hand, implies that executives provide a boilerplate answer.

To give you an idea here is an example of a boiler-plate answer by an executive which generates a high HAID score:

Question: “And it's nice to see that automotive ticking back in there again. Is the bid activity in automotive improving, or is it just kind of steady here?”

Manager A: “Automotive has been improving significantly. And as you know, we've reshaped a couple of our portfolios to pursue the more high-margin activity, the higher gross margin activity in our products. So, we have captured some solid automotive projects and we are continuing to bid several. So, we're seeing some positive trends in automotive.”

ChatBot A: “Yes, we're definitely seeing a steady improvement in automotive activity. We have a very strong product portfolio in the automotive sector, and we have been seeing steady bid activity as more and more auto manufacturers look to meet global emissions standards. We feel very good about our position in this market and the potential for continued growth going forward.”

What the research found was that executives tend to hide behind boilerplate answers more when they are delivering bad news. If they are discussing problems at their business, they obviously don’t want to go into too much detail about how bad the situation really is.

But more importantly, whenever executives provided additional insights and differed more from the AI-generated answer, analyst recommendations got better, and analyst forecast errors declined. Furthermore, trading volume increased by 14.6% as investors were better able to digest information and more confident in their investment decisions. And this also then increased abnormal returns in the shares. Positive news disclosed increased abnormal share price returns by some 2.6% which is quite a big reward for being transparent and informative in an earnings call, if you ask me.