Trust but falsify in action

Hello, and welcome back to my daily musings. Did you miss me? If not, let me start with a really depressing topic so that we get back into the right groove. Before I went on my summer break, I published a series of posts on how to do investment research. The fourth instalment of that series was titled “Trust, but falsify” and I explained why we should pay more attention to falsifying theories rather than simply treating violations of theories and models as ‘anomalies’. As it so happens, there seems to be an extreme case of ‘trust, but falsify’ going on in the field of Alzheimer’s Research that can teach us a lot about investment research.

Most of us know by now that one of the symptoms and possible causes of Alzheimer’s Disease is the build-up of ‘plaques’ in the brain tissue. This plaque is caused by the protein Amyloid beta (Ab) and most Alzheimer’s drugs try to target this protein build-up. Strangely, though, Alzheimer’s drugs have so far had extremely low success rates in clinical trials. Essentially, close to 100% of all Alzheimer’s drugs have been unsuccessful so far.

In what I consider a bombshell article, Science Magazine published the concerns of Matthew Schrag, a neuroscientist at Vanderbilt University, in July. It is well worth reading the whole article, but in a nutshell, Schrag found several ‘red flags’ in a foundational paper published in Nature in 2006 by Sylvaine Lesné from the University of Minnesota, Twin Cities. This paper established that a specific form of the Ab protein, called Ab*56, caused dementia symptoms in rats. Since then, the vast majority of Alzheimer’s tests have tried to measure this protein in humans as an indicator of Alzheimer’s. Similarly, the majority of Alzheimer’s drugs target this protein to treat or reverse the symptoms of the disease.

Schrag’s red flags indicate that the original findings may have been tampered with and may be the result of fraud. Now, let’s be clear that there is no clear evidence of fraud, yet, and the authors of the original article in Nature are presumed innocent until proven guilty, but if this paper was fraudulent, then we have essentially wasted the better part of 16 years of Alzheimer’s research and the biotech and pharmaceutical companies that are developing Alzheimer’s drugs based on this protein will likely fail (check your portfolio holdings, I’d say).

The reason I bring up this case is because it shows how science is supposed to work. People publish findings in peer-reviewed journals and once published, other scientists are building their work on these results. But once there are indications that the original findings may not hold in practice, the entire machinery to check results and reverse previous findings goes into action. Previously published research is scrutinised, and people try to replicate the original findings. If they find no errors and can replicate the findings, that’s fine. If they find either wrongdoing or the original findings cannot be replicated, the results are taken seriously. If the 2006 paper in Nature turns out to be fraudulent, people will not hesitate to throw 16 years of research overboard and start again.

Now compare this to finance and economics research. The people who developed the CAPM or various theories and models in economics and finance did do nothing wrong. They weren’t fraudsters. They developed the best theories and models of how the world works based on the knowledge and the data available at the time.

But were economics and finance failed is once data was presented that showed that these models were violated in practice. Instead of trying to come up with better models, these violations were branded ‘anomalies’ and the creators of the different economics and finance theories came up with all kinds of reasons, why their theories should hold anyway. Think about how long it took behavioural economists to overthrow the assumption of the rational agent, homo economicus. There are still some economists who defend this construct as practically relevant.

Or think about people like Margaret Thatcher’s favourite economist Patrick Minford. He apparently is still stuck in the 1980s world of supply side economics despite the massive evidence that tax cuts don’t boost economic growth or pay for themselves (see here for an analysis of tax cuts and their impact on GDP growth and here for an analysis of the impact of tax cuts on deficits). Minford’s views on deregulation and supply side economics are so extreme that they have been the subject of public rebuke by other economists (see here for an analysis of Minford’s predictions on deregulation). He is also the only economist who predicted that Brexit would boost economic growth. An analysis that has so irritated economists that a group of them felt obliged to set the record straight on Minford’s assumptions.

Yet, despite being discredited and shunned for the better part of three decades now, Minford is still quoted by investors and fringe economists. And Conservative Party leadership candidate Liz Truss has again pointed to Minford as the person who supports her plans for aggressive tax cuts. In economics and finance, these long-discredited ideas are just not going to die. And that is why we are wasting time, energy and money instead of developing better models of the world that could boost investor returns, improve welfare and reduce unemployment.

I pick here on Patrick Minford only because he is again in the headlines and his outsized influence on UK politics has, in my view, caused a lot of damage to this country. But Minford is only one case of many. Think of asset managers and investment consultants who build portfolios using the CAPM as a starting point to calculate expected returns for different assets. Think of left-leaning economists who argue that reducing the number of hours worked per week increases employment or that higher wages boost economic growth. Or think of monetarists who argue that ‘inflation is always and everywhere a monetary problem’. All of these theories and models of the world have been empirically falsified, yet they continue to influence our investment portfolios and economic policies. The damage done by the inability of economics and finance to falsify theories and then move on to develop better models and theories like they do in medicine is enormous in my view and we all need to do better in service of our clients and society at large.