A boost to productivity growth from AI?

I want to conclude this little series of posts on productivity growth that I have been writing for the last three weeks by focusing on a development that gets many people excited these days: AI and especially generative AI like chatGPT and Bing AI. Now, we have seen so many technologies being touted as a key driver for stronger productivity growth in the past and none of them have lived up to their promise, so you will be forgiven for being sceptical about the current hype, but let me go on the record that I think this one could indeed be different and there is some early data that shows generative AI may indeed create a lasting increase in productivity.

First, let me state that I am not an expert on chatGPT, Bing AI and the other AI applications that are being rolled out. If you want to learn how to use them most effectively and the pitfalls inherent in them, I highly recommend Ethan Mollick’s blog “One Useful Thing”, which has become my one-stop shop on all things AI. I also found the three studies I will discuss today on his blog, so full credit to him for pointing these out.

While I am not an expert on these tools, I have become a frequent user of Bing AI in both my personal and my professional lives. I have become such a fan of Bing AI that I have essentially abandoned Google’s search (with the notable exception of Google Scholar) and now use Bing as my go-to search engine. In my experience, Bing AI has made my professional duties much easier as well, relieving me of some tedious tasks and increasing the speed with which I can collect useful sources related to a specific research topic. I still have to spend time checking the sources and their accuracy (never trust an AI to come back with accurate data), but my time saving on mundane tasks have been large and are getting bigger the more I learn to use these tools.

To get onto the finishing stretch of this series on productivity, let’s take a look at the UK’s productivity growth over the last 150 years taken from the database of the Bank of England. I show 10-year average productivity growth rates for total factor productivity and labour productivity (GDP per hour worked). Clearly, the last twenty years have been abysmal with productivity growth declining towards zero.

Productivity trend growth in the UK

Source: Liberum, Bank of England

But if we look at the chart, we can also identify certain periods of higher productivity growth that lasted for a decade or more. During the 1870s, the British Empire was at its peak reaping the combined benefits of a global colonial resource pool and the wide adoption of steam engines and other machinery to automate production processes. Another small productivity boost came with the adoption of electric lights in the late 19^th century. But the boom time for productivity growth was the period from 1920 to 1970. During these five decades we saw the widespread adoption of the internal combustion engines in the form of cars, airplanes, and ships. We also saw the widespread adoption of modern communication technologies like the radio and TV as well as intercontinental communication via subsea cables and satellites.

But since the 1970s, it’s essentially been a constant decline, interrupted only briefly by the adoption of personal computers and data processing in the 1980s and the internet in the 1990s. But as you can see, these two technologies have provided only a small boost to productivity for a limited time and a priori, my attitude on AI would be that the productivity boost from this technology would be similar in nature to these two predecessors.

This is where two new studies on the productivity increase from AI adoption are beginning to change my mind and make me much more optimistic about productivity gains from this development.

The first study by Sida Peng and co-authors asked professional programmers to complete a programming task. Half of the participants were allowed to use GitHub Copilot – a generative AI – to complete their task, the other half was not. The programmers who used the AI assistant completed their tasks on average in 71 minutes, compared to 161 minutes for the programmers without AI support. That is a 55.8% reduction in time to complete the task.

Time to complete a programming task with and without AI assistance

Source: Peng et al. (2023)

That is an enormous increase in productivity. For comparison, in the first chart of UK productivity growth above, that is the change in productivity we experienced between 1920 and 1950. Thirty years of high productivity growth potentially triggered by one new technology!

Of course, software programming is just one small area of the economy but obviously one of growing importance. And while there is some creativity involved in coming up with an algorithm, most of the work is editing, debugging and optimising the algo, rather than actually coming up with it. And the time taken for these tasks of editing and debugging can be massively reduced to free up time for more creative tasks.

A second study gave some 200 university-educated professionals a writing task related to their job (these professions ranged from HR professionals to consultants and marketers). Half of the participants were allowed to use chatGPT to complete the task, the other half was asked to complete the task without the help of the AI. Again, the time savings were large with the average time taken to complete the task reduced from 27 minutes to 17 minutes (a 37% increase in productivity). The resulting texts were then evaluated by independent referees and the average grade for the tasks completed with the help of AI increased by one notch on a seven-digit scale. To put it in more familiar terms, the grade on average improved from a C+ to a B.

Another finding of the second study was that weaker participants benefited more from the use of chatGPT. The quality of output and the time taken to complete the task did not improve a lot for the top performers, but the weakest performers saw large improvements. This indicates that chatGPT and similar tools are predominantly a leveller of quality, bringing underperformers to a higher level of quality. The good news for all the outperformers among my readers is that the people who produced outstanding work still outperformed after the introduction of chatGPT, the gap to the underperformers was just smaller.

Finally, the second study also showed that the amount of time consumed for different elements of a task is changing. Without chatGPT, people use most of their time to draft a text that then is edited to the finished product. With the help of chatGPT, the time to come up with a first draft is reduced significantly, but there is more effort needed to edit this first draft. That is also my experience of using Bing AI (i.e. chatGPT linked to the Bing search engine). I can come up with a first shot at a summary of sources or trends in an investment area much quicker than before, but I need to edit the results generated by the AI quite a bit more than a draft I wrote myself from scratch.

Time allocation for tasks with and without chatGPT help

Source: Noy and Zhang (2023)

These are only first studies and we don’t know if the productivity gains achieved in these experiments will really come through in real life. After all, organisations have a habit of using productivity gains in one area to increase admin and bureaucracy in another, thus eliminating much of the productivity gains.

What is clear is that generative AI like chatGPT has wide implications and can significantly impact jobs that we so far thought were largely immune from automation and AI. Professional jobs in the service sector can be transformed forever, providing the opportunity for large increases in productivity growth in an economy-wide setting. To this extent it is worthwhile reading a study by Edward Felten and others on how chatGPT can transform a wide range of jobs. I have taken a selection of prominent jobs from their list and show the estimated disruption of these jobs from AI from biggest disruption (telemarketers, which can essentially be replaced by chatbots) to the least disruption (pressers and garment workers since chatGPT is really bad a sewing a dress).

Estimated disruption by chatGPT & co.

Source: Felten et al. (2023).

In summary, I am excited about generative AI and look forward to the next ten years or so when we see how much this technology will change our lives. I hope it will create a long-lasting boost in productivity growth, but so far, I am not sure. But it is worth a try.