The universities we attend append Bachelors of Science and Masters of Science prefixes to courses in economics. News headlines would convince you that the work of economists can be taken as gospel. Decision-makers rely on the advice of economists to make “evidence-informed decisions”.
Economics is usually treated as a science. But should it be?
What is a “science”?
The Oxford Dictionary definition for science as a discipline is:
The systematic study of the structure and behaviour of the physical and natural world through observation, experimentation, and the testing of theories against the evidence obtained
Now, someone defining something as thus doesn’t make it so. However, for the purposes of this article it captures what I consider makes something a science: a systematic approach to testing hypotheses about the world, and learning from the evidence.
I want to clarify that I’m not questioning the “scientific-ness” of all economics. There has been plenty of excellent science done in the field of economics, particularly in microeconomics1.
What warrants more consideration though is the “scientific-ness” of macroeconomics. Specifically, I am questioning the reliability of macroeconomic forecasts - particularly forecasts relating to Gross Domestic Product (GDP)23. As a result, I am also questioning the emphasis we put on GDP forecasts generally - the time spent developing them, their coverage in the news, and their influence on our decision making.
How do GDP forecasts perform?
To summarise: pretty poorly.
The Irish Central Bank is THE monetary authority in Ireland. You would like to think they understand what is going on in the economy. Yet, their predictions for the subsequent year’s economic performance are consistently wrong. Not only slightly either - often by quite a lot. In fact, sometimes they predict growth, and recession materialises (or vice versa).
In fairness to the economists at the Central Bank, all of the Quarterly Bulletins I reviewed constantly caveat their predictions with uncertainty (though this is often unquantified). They also provide scenarios on certain occasions (such as for Brexit and Covid-19).
Nonetheless, these Quarterly Bulletins exist, and within them is a forecast for the following years GDP growth. So, it would seem only fair that we compare the performance of these forecasts with actual GDP growth4.
Two observations stood out to me:
Firstly, there is considerably more “noise” in actual GDP growth compared to forecast GDP growth (see Figure 1). What I mean by this is that the forecast GDP growth line is much more smooth.
Secondly, the Central Bank forecast that the economy would shrink in 2 of the 19 years between 2005 and 2023. It actually shrunk in 4 of those years. It only correctly predicted that the economy would shrink in one year - 2009.
Lets put some figures on that noise I mentioned above. The Mean Absolute Deviation of forecast GDP from actual GDP was 4.81%. The Median Absolute Deviation was 3.48%.
In plain English, please? Well, in the case of the Median Absolute Deviation, it means that a GDP growth forecast for any given year deviates from actual GDP growth by ±3.48% on average. Given the magnitude of GDP growth forecasts examined here - which range from -2.3% to 7.3% - this is a very large level of deviation.
Suppose I was an economist trying to make a prediction about GDP based on data from 2005 to 2023, what would these deviations mean for me? Figure 2 shows us just. The shaded blue area around the forecast line provides a range of uncertainty (margin of error5) for forecast GDP growth.
Almost immediately you can see that the margin of error for 6 of the 19 years includes a figure less than 0%. In other words, a forecast would be so uncertain that it wouldn’t be possible to tell whether the economy was going to grow or shrink6. Hardly anything to hang your hat on.
You can also see that actual GDP growth is outside the range of forecast GDP growth in 9 of the 19 years. This again reflects the level of noise in actual GDP growth in comparison to forecast GDP growth.
So what?
If GDP forecasts are so inaccurate, why do we bother with them?
In my opinion, the main reason is the perceived alleviation of uncertainty. The future is inherently uncertain. We decide to ignore this, thinking that we can predict it. We do this to soothe the dread that uncertainty entails. A “scientific” forecast about the future offers us some respite - something we can lean on when making decisions. In reality though, GDP forecasts tell us very little about the future state of the economy.
But does it really matter if they are inaccurate? Does it matter that we spend time trying to predict the unpredictable? Does it matter that people might use these inaccurate forecasts to make decisions? I would argue that yes, yes it does:
Predicting the Unpredictable: Often, forecasts are linear extrapolations of some current trend. In other words, they often say “if things continue as they are this might happen…”. What’s the problem with that logic? Well, things aren’t guaranteed to continue as they are. Consider just some of the major world events that have occurred in the last 20 years highlighted in Figure 2.
What forecasts might tell us though, is what macroeconomists think about the economy. Maybe there’s some value in understanding that? Consider the 2008/09 Financial Crash illustrated in Figure 2. In 2007, macroeconomists obviously felt overly optimistic about the economy. By 2009 they evidently felt more pessimistic than actual economic performance warranted in 2010.
I am personally skeptical of the usefulness of macroeconomic forecasts, but that doesn’t mean others are. The problem is that this can lay shaky foundations upon which to base decisions.
Fragile Foundations: What happens then when we treat the prediction of the unpredictable as a science? We create an illusion of validity. We attach irrational credibility to these predictions, and they can lead decision makers astray.
Now it would be very difficult to measure the costs of inaccurate GDP forecasts, but we can at least imagine some of the potential negative impacts. Suppose I am a Government official who decides to reign in healthcare investment in anticipation of an (incorrectly) predicted economic downturn? Or if I was a CEO who prematurely made employees redundant in anticipation of an (incorrectly) predicted recession? Or if I stretched my liquidity too far in anticipation of (incorrectly) predicted economic growth next year?
The treatment of macroeconomic forecasts as scientific implies that macroeconomists understand the dynamics of an economy. In reality, they don’t. GDP forecasts tell us as much about the future as…well, divination.
What should people do instead?
I apologise for my vagueness here, but this will hopefully be the topic of (many) future posts. In general though, I would consider the following as guiding principles.
Spend less time in trying to predict the future (in situations where you can’t), spend more time planning for what should be done if certain scenarios arise7 and develop systems that are at least robust, if not anti-fragile8 (borrowing Nassim Nicholas Taleb’s term here).
Acknowledgements
The loved ones subjected to my ramblings yet graciously offering to read my drafts.
Hemingway was used to simplify some language.
Methodology
Data Sources
The following data sources were used as part of this article:
Irish Central Bank Quarterly Bulletins (Note: The Quarterly Bulletin prior to the forecast year is used for the Forecast GDP change. For example, for 2006 Quarterly Bulletin Q4 was used)
CSO Gross Value Added at Constant Basic Prices by Sector of Origin and Gross and Net National Income at Constant Market Prices (NA006)
The following table details the series of forecast and actual GDP change data used for this article:
Calculations
The following table details the deviation between forecast and actual GDP growth calculated for each year considered in this article:
The following table details a selection of summary statistics mentioned throughout the article:
Though there has also been questionable science done in the field of microeconomics too. I assume this is the case with every field though really.
There are other types of macroeconomic forecasts of course. However, for this article I examine GDP as it is generally considered a “key indicator” of economic performance (which in itself is worthy of debate, but would require an entire post on its own).
I am aware that Irish GDP is inflated dramatically by the location of intangible assets (primarily intellectual property) in Ireland (see CSO Note on Irish GDP in 2015). In 2017 the CSO released a measure that is believed to better reflect actual activity in the Irish economy - Modified Gross National Income (GNI*). However, for the purposes of this article I will persist with GDP for several reasons:
GDP is seen as a “key indicator” of economic performance internationally (see Footnote 2)
As far as I could see the Central Bank do not provide a direct forecast of GNI*. If someone finds this though I would gladly investigate.
I don’t believe (though I could be incorrect) that there would be much of a difference between the accuracy of a GDP prediction and a GNI* (though that is, of course, an assumption that benefits my narrative here, and isn’t a piece of evidence in favour of my argument).
See “Methodology” section for more detail on the data used and…well, the methodology.
See Footnote 4
A caveat that this is based on the deviation of forecast from actual growth for 2005 - 2023, not the level of uncertainty that the Central Banks models likely provides (but is rarely shown).
The broad umbrella-term for this type of work is “Futures Thinking”. “Strategic Foresight” is a particular tool in the Futures Thinking toolbox which is used to created strategies for the future.
An anti-fragile system is one that benefits from mishandling; one that benefits from a necessary level of stressors to produce positive change. These types of systems thrive on randomness, disorder and volatility. For more on this read Nassim Nicholas Taleb’s Antifragile
Interesting article Shane! Funny how until 2013, forecast GDP tracks the actual GDP quite well. I wonder did the migration of aircraft leasing and tech companies to Ireland distort our GDP beyond what economists could reasonably measure!