Bad Data: How Governments, Politicians and the Rest of Us Get Misled by Numbers

Georgina Sturge is a statistician at the House of Commons Library, an in-house research service for all members of the UK Parliament in Westminster. As an experienced statistician working for the UK government, her role involves frequently communicating data with politicians. She has shared her valuable insight on how often politicians find themselves in tough spots as a consequence of ill-informed policymaking based on bad data. And how such misguided policies take a toll on human life and economy of a state. In doing so, she has taken the UK as a case study to elaborate on her point.
The book questions the authenticity of data used/provided by politicians, statisticians, economists, and academicians and advises care on data consumption regardless of its source. The limitations of data availability and accuracy persist: firstly, because of impossibility of counting and measuring certain things. Not everything has a fixed definition, such as poverty, gender equality, and racism are a few such concepts where it is difficult to set the indicators. Secondly, the data-collecting process apart from being painstaking is also susceptible to intentional and unintentional human errors in ways seldom considered by data consumers. Surveys, she explains, mostly rely on human assumptions, prior knowledge, and reasoning — which is to say, they involve questions that already presume what truth might be. These underlying facts encourage the use of bad data in decision-making because politicians are always under pressure to yield concrete numbers on short notice. In such scenarios, bad data conveniently gets incorporated into hasty decision-making.
One among the plethora of other examples she gives in her book is the infamous UK’s 2011 census when it caught a big blunder in data collection regarding number of migrants. It found that compared to pre-census estimations, more than half a million migrants were living in the UK. Such a big number was missing from the official estimates because bad estimates were made on migrants through the UK’s official International Passenger Survey. In the 2019 census, the same survey was found to highly overestimate the number of migrants, which polarised political opinion on immigrants and influenced citizens’ opinion during the EU Referendum when forty-six per cent of people in Britain thought of immigration as the most important issue in the country. In this context, she reemphasises the dubious nature of data.
However, in identifying the gaps in data collection methods, she does not completely disregard the utility of data accumulated through surveys and other methods. As she points out availability of some data, even limited, is better than no data provided that this limited data is accurate. Since the loopholes attached to the design choices and models used in deciding what to count and measure are subtle and mostly invisible, they can distort the truth altogether and cause big blunders in policymaking. Economics, Sturge declares, is an inexact science like behavioural psychology, which is why none of the world’s leading economists saw the 2008 financial crisis coming. However, economists are often very much confident about their models, and politicians have to rely on them in policymaking.
The book is not a one-sided critical take on bad data, but it also gives examples of good data and the ways to extract such data. For instance, she argues that the recorded changes in data measured over a longer period tend to become more reliable since adjustments are often made to collect the same data in better ways. Also, despite the feeble nature of data, the author is still hopeful that prudent use of big data and artificial intelligence in future policymaking offers great opportunities, even though currently the data fed into them is half-baked. However, she also acknowledges that use of digital algorithms in today’s world has amplified the chances of disasters in policymaking on a much bigger level.
Sturge comes off as very clear in her take on the uncertain nature of data. As a statistician, she acknowledges and endorses the uncertainty and U-Turns taken by politicians rather than their rigidity in decisions based on out-dated data. She compares UK former Prime Ministers Margaret Thatcher and Ted Heath to explain how people tend to appreciate the rigidity of opinions in politicians instead of their flexible attitudes in policymaking. Thatcher being rigid in her decisions was always celebrated as an ‘Iron Lady’ during her tenure, unlike Ted Heath, who was flexible and thus considered weak and wobbly. It is the comparisons and examples like these that make for compelling arguments that she makes in her entire book.
However, the language might appear assertive as one approaches the end of the book. She goes all out to criticize politicians, researchers, statisticians, and producers of data who give the impression of omniscience and project confidence in their conclusions when, in reality, there is no clear blueprint of knowing what works when it comes to, say, for instance, reducing crimes.
Overall, the book is a quick and informative read mostly aimed at the general public so they can understand the intricacies associated with the accuracy of data and statistics, which they often tend to ignore. For this purpose, the book is written in the language of laymen and is packed with so many brow-raising examples of where policymaking went wrong based on bad data and how can such instances be avoided. For the most part, Sturge does manage to poke into readers’ own biases and ignorance being consumers of data as she reveals blunders made in the closed-off world of policymaking that resulted in disastrous consequences.