Publication bias in economics

On 27 October, I wrote about a study that showed that there is no link between corporate tax rates and economic growth in a country. As part of that post, I stated that publications that found a positive relationship between lower taxes and higher growth were 2.7 to 3 times more likely to be published than studies that found a negative relationship. Some people asked me if the bias in published papers could really be that big? The answer is it may even be bigger.

Frantisek Bartos and his colleagues analysed 26,000 meta-analyses of research in medicine, economics, and psychology. The chart below compares the evidence they found for a publication selection bias (PSB). The way to read this is that the larger the area under the curve in each field, the more studies were subject to publication selection bias. It is clear that the problem is much, much worse in economics than it is in medicine or psychology. The probability of a study finding an effect in economics is 99.9%(!), but if publication selection bias is taken into account, this drops to a mere 29.7%. In other words, studies that find a significant effect are on average 3.4 times more likely to be published than studies that find no effect. In psychology, the probability of finding an effect in the published literature is 98.9%, but it drops to 55.7% if publication selection bias is taken into account. In medicine, finally, the probability of finding an effect in the published literature is 38.0%, dropping to 27.5% after publication selection bias is taken into account.

This is dramatic, also for the thoughts I publish here because I use academic papers to justify my views. Turns out that in all likelihood two out of three findings I report on are likely not real. Excuse me please, while I withdraw into my room and cry…

Evidence of publication bias in medicine, economics, and psychology

Source: Bartos et al. (2022)