Summary: Adults make fewer errors in judging statistical information when they are able to experience the information firsthand, a new study reports.
Source: Max Planck Institute
Babies have the intuitive ability to make correct statistical judgments from an early age and apes are also surprisingly savvy when it comes to probabilities, as recent studies in the fields of early childhood cognition and behavioral biology have shown. Adults, in contrast, are often seen as poor intuitive statisticians.
How can these contradictory findings on intuitive statistical reasoning be explained? Do humans lose the ability to make intuitive probability-based inferences as they grow up? Or does it have something to do with the way in which the information is presented? Two new studies provide insights into these questions.
In a first study, researchers from the Max Planck Institute for Human Development and the University of the Balearic Islands analyzed the development of research on statistical reasoning from a historical perspective. In the 1960s, adults were still considered to be good intuitive statisticians.
From the 1970s, however, this notion was overturned. Since then, studies on so-called “heuristics and biases” have concluded that human reasoning is prone to systematic errors. According to this approach, the heuristics—mental shortcuts or rules of thumb—that people use for statistical reasoning produce systematically biased judgments that are inconsistent with the rules of probability theory and statistics.
What has escaped attention until now, however, is that there was also a fundamental change in the design of the studies used to test adults’ statistical reasoning. Since the 1970s, most of these studies have involved text-based tasks and descriptions. Experience-based tasks, in which participants were able to experience statistical information at first hand, became a thing of the past. Yet the new study is the first to draw a causal connection between the dramatic change in experimental methods and the fact that findings of poor statistical reasoning in adults have been accumulating ever since.
What does this finding mean for future studies on human statistical reasoning? Should researchers return to the tradition of experiential studies and avoid text-based descriptions? “In practice, we can’t work exclusively with experience-based tasks, but we shouldn’t rely solely on description-based tasks either. We need to be aware that the two methods produce qualitatively different results. Ideally, both should be used in combination,” says Tomás Lejarraga, Associate Research Scientist in the Center for Adaptive Rationality at the Max Planck Institute for Human Development and Director of the Decision Science Laboratory at the University of the Balearic Islands.
To further examine the relationship between task format and performance, in another study researchers from the Max Planck Institute for Human Development compared numerous publications on the statistical intuitions of very different groups of participants. They focused on babies and nonhuman primates—two groups that recent studies have shown to be surprisingly capable of statistical learning and inference.
Whereas studies tend to use symbolic, abstract descriptions to assess adults’ statistical reasoning, infants or animals have to experience the statistical information at first hand by interacting with the environment—after all, they cannot read.
For example, one way of testing babies’ statistical intuitions is for an experimenter to draw a colored ball from an opaque box of balls. The content of the box is then revealed, and the length of time the babies spend looking at the balls in the box is recorded.
Babies tend to look for longer when the sample drawn does not reflect the color distribution of the balls in the box than when the sample is consistent with that distribution. This suggests that babies already have a basic understanding of random sampling. Similar studies have been conducted with animals by presenting them with food items.
Adults’ decisions have been shown to improve when they are able to experience probability information themselves—for example, when they play a lottery repeatedly. Studies have also shown that adults make fewer errors in judging statistical information when they are able to experience that information at first hand—for example, in a computer simulation.
“Awareness of this description–experience gap in statistical reasoning is of major relevance for education policy and the development of teaching methods. For example, schools could teach statistics and probability by means of hands-on examples and simulations, rather than primarily through written tasks,” says Christin Schulze, Senior Research Scientist in the Center for Adaptive Rationality at the Max Planck Institute for Human Development and lead author of the study comparing findings from research on babies, nonhuman primates, and adults.
Although the two studies take different approaches, their results are consistent: It makes a difference whether we learn about probabilities from descriptions or from experience.
“Many of our everyday decisions have to be made under uncertainty, with the help of statistical intuitions. Our findings show that the statistical intuitions of primates, infants, and adults can be surprisingly good. The key factor is how we encounter statistical information. These findings indicate that our statistical intuitions are by no means as irrational as has long been suggested,” says Ralph Hertwig, Director of the Center for Adaptive Rationality at the Max Planck Institute for Human Development.
A description–experience gap in statistical intuitions: Of smart babies, risk-savvy chimps, intuitive statisticians, and stupid grown-ups
Comparison of different lines of research on statistical intuitions and probabilistic reasoning reveals several puzzling contradictions.
Whereas babies seem to be intuitive statisticians, surprisingly capable of statistical learning and inference, adults’ statistical inferences have been found to be inconsistent with the rules of probability theory and statistics. Whereas researchers in the 1960s concluded that people’s probability updating is “conservatively” proportional to normative predictions, probability updating research in the 1970s suggested that people are incapable of following Bayes’s rule. And whereas animals appear to be strikingly risk savvy, humans often seem “irrational” when dealing with probabilistic information.
Drawing on research on the description–experience gap in risky choice, we integrate and systematize these findings from disparate fields of inquiry that have, to date, operated largely in parallel. Our synthesis shows that a key factor in understanding inconsistencies in statistical intuitions research is whether probabilistic inferences are based on symbolic, abstract descriptions or on the direct experience of statistical information. We delineate this view from other conceptual accounts, consider potential mechanisms by which attributes of first-hand experience can facilitate appropriate statistical inference, and identify conditions under which they improve or impair probabilistic reasoning.
To capture the full scope of human statistical intuition, we conclude, research on probabilistic reasoning across the lifespan, across species, and across research traditions must bear in mind that experience and symbolic description of the world may engage systematically distinct cognitive processes.