Do you trust AI forecasts?

Analysts are using AI more and more as part of their research process. In some cases, AI is also involved in making forecasts. But should analysts tell their clients that they use AI to make these forecasts?

Francesco Stradi and Gertjan Verdickt recruited some 3,400 Americans and asked them to give a forecast for the S&P 500 return over the coming twelve months. Then they confronted them with one of three statements by an expert and measured, how much the participants updated their own estimates.

The three statements were:

• Human: Analysts from Goldman Sachs forecast a 12-month return of 5% for the S&P 500 index. The baseline assumption during the next year is that the US economy will continue to expand at a modest pace and avoid a recession.

• Human + Machine: Analysts from Goldman Sachs, incorporating advanced AI tools to enhance their analyses, forecast a 12-month return of 5% for the S&P 500 index. The baseline assumption during the next year is that the US economy will continue to expand at a modest pace and avoid a recession.

• Machine: An advanced AI model, trained on financial data and market trends, forecasts a 12-month return of 5% for the S&P 500 index. The baseline assumption during the next year is that the US economy will continue to expand at a modest pace and avoid a recession.

The chart below shows the average credibility rating the participants in the experiment give to the three different statements.

Credibility rating of human, machine, and human + machine forecasts

Source: Stradi and Verdickt (2024)

As you can see, humans trust other humans slightly more than machines or a combination of human plus machine. However, this bias against machines is not universal. People with higher AI literacy or higher investor sophistication trust machine forecasts more than human forecasts.

There tends to be a minority of investors who are sceptical about the many flaws of forecasts made by analysts out of flesh and blood that they prefer to listen to forecasts made by bits and bytes instead.

My personal experience rhymes with these results. In my experience there are research analysts who are more quantitatively oriented. These analysts tend to use models and more sophisticated analysis to come up with recommendations and forecasts. I am one of them. And then there are other analysts who use more qualitative arguments and ‘back of the envelope’ analysis to come up with their views.

It’s not that one approach is better than the other (well…) but, in my experience, younger and mathematically more sophisticated investors will tend to gravitate towards analysts that provide more quantitative analysis while more traditional investors will gravitate more towards traditional, more qualitative analysts. Horses for courses, I guess.