It just wouldn’t be a discussion of how numbers can mislead without an invocation of the famous quote falsely attributed to Mark Twain, who falsely attributed it to Benjamin Disraeli, which states that “there are three kinds of lies: lies, damned lies, and statistics.”
I like to think that the actual author, Charles Dilke, might have found some humor in the misattribution of those words. I also like to think that Dilke, who died over a hundred years ago, would be able to offer some interesting perspective on the data-driven world we now purport to live in today.
While there are many positive impacts that the prevalence and accessibility of quantitative data has had on public policy in the last few decades, the idea that “data” — one of the more dangerously vague words that I regularly (quite regularly) encounter — paints a singular, factual portrait of the world is patently ridiculous. Instead, the world of “data”, while no doubt rich with moments of insight, is a veritable goldmine of misinterpretation, contradiction, and confounding evidence. And unlike other methods of mistruth, numbers and their visual representation have an uncanny ability to convince people. Put simply: numbers lie, and they lie well.
I’ll leave it to the psychologists, mathematicians, and screenwriters to press my case.
Now, I am not suggesting that this is a reason to abandon “data” as a tool for policy and decision making. (Much of my work is focused on incorporating data into decision making!) But it is every reason to be critical about the data that one consumes and cautious as to the story it tells.
Let’s explore a topical example. Consider the following two statements:
Statement 1. During the Great Recession, the black unemployment rate grew more rapidly than the white unemployment rate.
Statement 2. During the Great Recession, the black unemployment rate was reduced as a percentage of the white unemployment rate.
I’ve seen and heard some version of each of these statements recently, from those seeking to apply lessons from the Great Recession to the current recession (which, surely you’ve heard, is now official). Both statements are factually true. But one is also a lie — because it uses its facts to deceive. (Technically, we might call this paltering.) Let me explain why I say that with an illustrated example.
First, let’s look at the US employment rates as reported by the BLS for the black labor force and the white labor force, spanning the Great Recession. In August, 2007, preceding the recession, black unemployment was 7.6% of the labor force and white unemployment was 4.2%. Following the recession in August, 2009, black unemployment had risen to 14.8% and white unemployment had risen to 8.9%.
Take the percentage difference between those two sets of numbers out of context, and you’ll find that the percentage difference in 2007 was greater than in 2009. (The ratio between 7.6 and 4.2 is greater than the ratio between 8.9 and 14.8.) But percentages within rates are mathematically dicey territory, and in this case, they lead to a starkly inaccurate statement. Consider a simplified illustration of the situation, shown below, in which the rates stated above have been rounded to the nearest integer and applied to two groups of 100 workers.
A more critical accounting of what took place during the Great Recession would suggest that black workers faced disparity preceding the recession, saw disproportionate impacts during the recession (with 7 black workers newly laid off out of 100 and 5 white workers newly laid off out of 100 during the recession), and therefore faced increased disparity in the labor force afterwards.
If that sounds nitpicky, I have failed to make my point, for that is a massive distinction in its implications on public policy.
A decade after the Great Recession, the same pattern of widening unemployment rate disparity began to emerge in last month’s jobs report from the BLS*, a fact that was far less reported than the overall decline in unemployment (driven by the decreasing white unemployment rate). Tomorrow morning, the BLS will release June’s report, which will provide an updated look at estimated unemployment rates by race.
As we explore how to measure the impacts of this current recession — through unemployment as well as other economic data — we must be thoughtful in how we digest that data to inform our understanding of the world.
*The BLS also announced that errors in the collection and reporting of data had led to a significant underestimate of the unemployment rate in both April and May, but it is unclear how this affects the racial gap.