Summary: Learning is optimized in computer models when there is an error rate of 15%. Researchers say the 85% accuracy rule may also apply to humans for optimal perceptual learning.
Source: University of Arizona
To learn new things, we must sometimes fail. But what’s the right amount of failure? New research led by the University of Arizona proposes a mathematical answer to that question.
Educators and educational scholars have long recognized that there is something of a “sweet spot” when it comes to learning. That is, we learn best when we are challenged to grasp something just outside the bounds of our existing knowledge. When a challenge is too simple, we don’t learn anything new; likewise, we don’t enhance our knowledge when a challenge is so difficult that we fail entirely or give up.
So where does the sweet spot lie? According to the new study, to be published in the journal Nature Communications, it’s when failure occurs 15% of the time. Put another way, it’s when the right answer is given 85% of the time.
“These ideas that were out there in the education field – that there is this ‘zone of proximal difficulty,’ in which you ought to be maximizing your learning – we’ve put that on a mathematical footing,” said UArizona assistant professor of psychology and cognitive science Robert Wilson, lead author of the study, titled “The Eighty Five Percent Rule for Optimal Learning.”
Wilson and his collaborators at Brown University, the University of California, Los Angeles and Princeton came up with the so-called “85% Rule” after conducting a series of machine-learning experiments in which they taught computers simple tasks, such as classifying different patterns into one of two categories or classifying photographs of handwritten digits as odd versus even numbers, or low versus high numbers.
The computers learned fastest in situations in which the difficulty was such that they responded with 85% accuracy.
“If you have an error rate of 15% or accuracy of 85%, you are always maximizing your rate of learning in these two-choice tasks,” Wilson said.
When researchers looked at previous studies of animal learning, they found that the 85% Rule held true in those instances as well, Wilson said.
When we think about how humans learn, the 85% Rule would mostly likely apply to perceptual learning, in which we gradually learn through experience and examples, Wilson said. Imagine, for instance, a radiologist learning to tell the difference between images of tumors and non-tumors.
“You get better at figuring out there’s a tumor in an image over time, and you need experience and you need examples to get better,” Wilson said. “I can imagine giving easy examples and giving difficult examples and giving intermediate examples. If I give really easy examples, you get 100% right all the time and there’s nothing left to learn. If I give really hard examples, you’ll be 50% correct and still not learning anything new, whereas if I give you something in between, you can be at this sweet spot where you are getting the most information from each particular example.”
Since Wilson and his collaborators were looking only at simple tasks in which there was a clear correct and incorrect answer, Wilson won’t go so far as to say that students should aim for a B average in school. However, he does think there might be some lessons for education that are worth further exploration.
“If you are taking classes that are too easy and acing them all the time, then you probably aren’t getting as much out of a class as someone who’s struggling but managing to keep up,” he said. “The hope is we can expand this work and start to talk about more complicated forms of learning.”
University of Arizona
Alexis Blue – University of Arizona
The image is in the public domain.
Original Research: Open access
“The Eighty Five Percent Rule for optimal learning”. Robert C. Wilson, Amitai Shenhav, Mark Straccia & Jonathan D. Cohen.
Nature Communications doi:10.1038/s41467-019-12552-4.
The Eighty Five Percent Rule for optimal learning
Researchers and educators have long wrestled with the question of how best to teach their clients be they humans, non-human animals or machines. Here, we examine the role of a single variable, the difficulty of training, on the rate of learning. In many situations we find that there is a sweet spot in which training is neither too easy nor too hard, and where learning progresses most quickly. We derive conditions for this sweet spot for a broad class of learning algorithms in the context of binary classification tasks. For all of these stochastic gradient-descent based learning algorithms, we find that the optimal error rate for training is around 15.87% or, conversely, that the optimal training accuracy is about 85%. We demonstrate the efficacy of this ‘Eighty Five Percent Rule’ for artificial neural networks used in AI and biologically plausible neural networks thought to describe animal learning.