A simple, yet powerful way to use ChatGPT to analyse earnings calls

I have recently written a post about how analysts can use chatGPT and other large language models to summarise earnings calls and crystallise the information in them. Now, thanks to the launch of chatGPT-4 Turbo and chatGPT-4o we can do even better. A team of researchers from the University of Münster and Washington University in St. Louis have just provided us with one of the easiest, yet impactful ways to use chatGPT for financial analysis.  

The big of advantage chatGPT-4 Turbo and later is that it can digest enormous amounts of texts and is no longer limited in character counts. This allows analysts to enter entire earnings call transcripts and ask chatGPT to analyse it. And thanks to chatGPT-4o these earnings calls can even be entered in audio format (presumably as a live stream).

Using this, the researchers did something very simple. Their first prompt was:

“Please read the following transcript of a Question-and-Answer session from the earnings conference call of company {firm} ({ticker}) carefully. Determine whether the Question-and-Answer session of this earnings conference call is ‘usual’ or ‘unusual’: If the Question-and-Answer session is classified as ‘usual’, state ‘usual’ without any justifications or further output. If the Question-and-Answer session is classified as ‘unusual’, state ‘unusual’ and provide a justification for this classification. Transcript of the Question-and-Answer Session: ‘{qa}’”

Then, they collected all the answers chatGPT gave as to why it thinks a specific earnings call is unusual and gave it another prompt:

“Please read the provided text file with justifications for unusual Q&A sessions from earnings conference calls carefully. What are high-level categories to identify unusual Q&A sessions? Make sure that each statement from the text file can be assigned to one of the categories.”

The result was a classification system of 25 types of unusual interactions on earnings calls. The table below shows all 25 how many US companies in a given quarter show unusual communication (N) at the median fraction of companies in the entire sample of all US companies that show unusual communication (Q50; 0.43 = 43%, etc.).

Frequency of unusual communication on earnings calls

Source: Beckmann et al. (2024). Note = N average number of firms with unusual communication per quarter, Q50 = median share of companies with unusual communication in the sample.

No surprise to me that the most common unusual communications were lengthy responses by executives and detailed discussions of non-financial topics because that is how executives try to cover up negative news.

But once that work is done, one can ask chatGPT to assess any earnings call along these 25 dimensions of unusual communication:

“Please read the following transcript of a Question-and-Answer session from the earnings conference call of company {firm} ({ticker}) carefully. Determine whether the Question and-Answer session of this earnings conference call is ‘usual’ or ‘unusual’ in the following {len(categories)} categories: {categories} For each category, state whether the Question-and-Answer session is ‘usual’ or ‘unusual’. If the Question-and-Answer session is classified as ‘usual’ in the respective category, state ‘usual’ without any justifications or further output. If the Question-and-Answer session is classified as ’unusual’ in the respective category, state ‘unusual’, print a ‘/,’ and provide a justification for this classification. Transcript of the Question-and-Answer Session: ‘{qa}’”

This prompt will provide a list of characteristics along which chatGPT thinks any given earnings call is unusual. And thus guide investors to dig deeper on that issue.

Here are some examples of the kind of analysis chatGPT provided:

• “There is a noticeable avoidance by management to provide specific details on the 6 global fitness redesign initiatives, despite being pressed by analysts.”

• “Management seemed unprepared to address the specifics of the sales guidance changes for Advance Pierre and Chicken, as indicated by the need to follow up after the call [24].”

• “There is a moment of conflicting information regarding the acquisition expenses for Esterline. Michael Lisman mentions higher-than-typical expenses due to the size of the acquisition, but Kevin Stein then says the fees are not different than expected, which Lisman confirms.”

Of course, investors and trader often notice these inconsistencies in real time which is why the share price return for companies with unusual communication around the call tends to be worse. But no investor can listen to all earnings calls all the time and this simple methodology provides investors with a systematic toolset to analyse companies of interest and assess the weak points of their earnings reports. Personally, I love it, but as a corporate executive I probably wouldn’t because chatGPT makes it much harder to get away with deception and distraction.

Difference in share price return in the two days around an earnings call

Source: Beckmann et al. (2024). Note: *** (**, *) denotes the statistical significance of the difference between unusual and usual communication at the 1% (5%, 10%) level.