Forecasts are hard, part 1

It is July. It’s hot outside and people complain about having a hard time thinking straight. So what better topic for a post than probability and statistics?

As investors, we constantly have to deal with probabilities. There is no investment that is guaranteed to make a profit, no matter what some people may say. Not even US Treasuries are guaranteed to make a profit because there is a very small probability that the United States government will default on its promise to pay its debt. The probability is so small that it is irrelevant in practice, but it is not zero.

So, for every investment we make, we have to deal with different possible outcomes and assess the likelihood of these different outcomes. What makes things worse is that in investments, we typically have to forecast possible outcomes of an infinite set of possible outcomes each of which is not independent of the other.

To show you how different these problems are, compare the following two examples.

First, here is an example of forecasting the probability of an outcome when events are independent of each other and the number of possible outcomes is limited:

You throw with two dice. Then you can throw any number between 2 and 12. Now, you can throw 12 only by throwing six twice. Similarly, you can throw 11 only by throwing 5 and 6 once each. Which of the following is correct?

• The probability of throwing 11 is the same as the probability of throwing 12.

• The probability of throwing 11 is twice the probability of throwing 12.

• The probability of throwing 11 is three times the probability of throwing 12.

Most people understand that there is only one way to throw 12 with two dice (6-6), but there are two ways of throwing 11 (5-6 and 6-5).

But now consider this more realistic example:

Suppose the Minister of Finance needs to forecast next year’s inflation. He asks two well-known experts to advise him. The first expert responds that there will be 1% inflation next year, and the second that there will be 2% inflation. The two forecasts are published in the press so that everybody knows about them. The minister then reflects. He realizes that the two experts know each other and that they base their forecasts on the same data sets. Also, from past experience, the minister has more trust in the second expert than the first. After considering the two forecasts he declares the Ministry’s forecast to be 2.25%. What do you think of this?

• A forecast larger than 2% is certainly possible, given the fact that the two experts know each other

• I would have expected a forecast between 1% and 2%. Why does the minister ignore his advisors?

• Such a counter-intuitive forecast would only rarely be reasonable

Most people would probably find the forecast of the Ministry to be wrong and expect it to be between 1% and 2%. But in fact, the Ministry of Finance was entirely rational and correct in making a forecast of 2.25%. The clue is to realise that if the two experts know each other and use the same data, they are not making independent forecasts. In fact, the expert who makes the low forecast of 1% inflation will likely influence the other expert who wants to make a higher forecast.

The expert with the higher forecast may have made a forecast of 2.5% inflation if he had made an independent decision. But because he knows the other expert who makes a much lower forecast of 1% inflation, he slightly lowers his forecast from 2.5% to 2% to be more in line with the other expert. Similarly, the other expert may have increased his inflation forecast from 0.5% to 1%. Thus, experts knowing each other and using the same data to make their forecasts involuntarily move towards a narrower range of forecasts. The Minister of Finance realises this and at the same time believes that the forecaster with the lower number is less reliable, so the Minister accounts for this effect by taking the forecast of 2% from the expert he trusts and adding 0.25% to adjust for the bias in that forecast.

This is key to why experts tend to miss recessions and episodes of high inflation. The media likes to criticise central banks for missing the scale of the inflation problem we encounter this year. But think about it. Central bankers use the same data as all private-sector economists to forecast inflation. And central bankers know many private-sector economists. In fact, many private-sector economists used to work at a central bank at some point in their careers. And private-sector economists constantly publish their forecasts for inflation. If you see your colleagues and peers forecast inflation between 2% and 5%, it is pretty likely, that you, too, will forecast inflation somewhere between 2% and 5%. Imagine the reaction if your forecasts are complete outliers. You would expose yourself to a ton of criticism.

I have severely criticized the Bank of England’s recent extreme forecasts but maybe they have just tried to do what I described above: debias their inflation forecasts from the group pressure among economists. If so, I applaud the Bank of England for its bravery. Or they simply got their forecasts wrong and didn’t even think about that effect because even trained economists are really bad at statistics. You decide.