During Canada’s long federal election campaign, punditry, forecasts, assurances and warnings have filled the air like leaves in a gale. But how good is humankind at predicting anything? The authors of a new book have an answer. Not very.
Darts and chimps
Philip Tetlock suspected he was onto something 30 years ago when he invited big-name pundits to participate in a research project on the accuracy of their forecasts. None agreed. As Tetlock, a University of Pennsylvania psychologist, and Ottawa author Dan Gardner explain in their new book Superforecasting: The Art and Science of Prediction, there was good reason for the reticence. “People tend to be terrible forecasters.” Tetlock finished that project studying other “experts.” He found that on average they were “as accurate as a dart-throwing chimpanzee.”
Foxes and hedgehogs
Some of us are vastly better than the majority at prediction. In explaining, Tetlock borrowed from the ancient Greeks to divide people into two categories: hedgehogs and foxes. Hedgehogs see everything through the lens of one big idea. Foxes consider many ideas. Accurate foresight, the authors write, is “the product of particular ways of thinking, of gathering information, of updating beliefs.” Superforecasters tend to be remarkably numerate. They tend to be news junkies. They are willing to change their minds. “Superforecasters are supreme foxes,” Gardner said. “They want to get as many different perspectives as possible.”
Sure and loud
In punditry (and campaigning), however, hedgehogs have an edge. They tell “simple, clear stories that grab and hold audiences” and comfort electorates. They use words like “furthermore” and “moreover” to bolster their one big message. (Foxes use words like “however” and “on the other hand.”) Merely being frequently wrong has not stopped blowhards from continuing to make predictions (or politicians to peddle myths). “You’re not punished for having a lack of nuance, a lack of depth or a lack of accuracy,” Gardner said. “Does it give the illusion of knowledge? That’s all that matters.”
The safest thing for pundits and campaigning politicians to do is couch their opinions and promises in vague language. “A fair chance.” “Serious possibility.” “Significant improvement.” It’s hard to say what any of that actually means. And whenever one reads or hears “may,” it is good to take note that “may not” is its silent partner. To Gardner, forecasts must have clearly defined terms and timelines. “A forecast without a timeline is absurd.”
Doubt is good
In predictions and campaigning, certainty is often seen as a virtue, Gardner said, when it should scream Beware! “Reality is infinitely complex. Uncertainty is ineradicable. Therefore the expert who speaks with grand certainty is probably blowing smoke,” he said. “You want to look for people who appreciate that there is another perspective, that there are counter-arguments, that there are reasons for believing that something different may happen. If they’re not taking into account those alternative perspectives, chances are they’re looking at reality through a single lens. And that’s a terrible way to see reality accurately.”
Twilight of the Gurus?
Over the past 20 years, awareness of cognitive psychology has exploded, as has the volume and availability of data. So Gardner is hopeful that consumers and voters will be less susceptible to the lure of simple, untested certainties. “There is something at work in the culture. People are beginning to realize that you really need to test things.” Which is why the current federal government’s nixing of Statistics Canada’s long-form census — at a time when he would “double, triple, quadruple” the agency’s budget — leaves him baffled. Though not lost for words.
Data, data and more data
“Lunacy! Idiocy! Asinine! Antediluvian! For them to be diminishing our statistical capacity at a time when the entire planet is moving toward greater and greater collection of data and more sophisticated data analysis is unspeakably silly,” Gardner says. “People always ask me, ‘Why are they cancelling the long-form census?’ If I could ask Stephen Harper to answer one question, I’d want him to answer that.”