79% of oft-cited statistics are total garbage
You know, we learn we remember 10% of what we read, 20% percent of what we hear, but 80% of what we actually experience. Or, wait, maybe it’s 20%. Or 30?
Of course, as many people know, this delightful little statistic has no backing in any sort of serious research—nor, indeed, could it:
…As Dwyer points out, the reported percentages are impossible to interpret or verify without specifying at least the method of measurement, the age of the learners, the type of learning task, and the content being remembered (p. 10). Despite the lack of credibility, this formulation is widely quoted, usually without attribution, and in recent years has become repeatedly conflated with Dale’s Cone, with the percentage statements superimposed on the cone, replacing or supplementing Dale’s original categories.
from Cone of Experience (PDF), entry in A. Kovalchick & K. Dawson, Ed’s, Educational Technology: An Encyclopedia. Santa Barbara, CA: ABC-Clio, 2003.
Several bloggers have likewise been struck by the curious disconnect between the popularity of this statistic and its relation to reality. Despite its readily apparent dodginess (We remember 90% of what we experience? So I perfectly remember everything I did for nine out of the last ten years?), people love quoting this thing.
So quote they do. And, since there’s no actual citation for this thing, the meme is free to mutate, which is actually kind of fascinating; the plot above shows the pattern in ghits for different versions of this same ‘principle.’
But why? Obviously, the meme lives because it has value to people; in this case, it helps folks prove a point about better ways of teaching. But that’s not really an answer; there’s no reverse version of this for people arguing the opposite side. No, the real answer is this: the statistic lives because it demonstrates something that the speaker and the listener both already agree on. Few people are going to call you on this statistic, because everyone knows that the gist is true in many situations; you probably will learn something better if you involve it in some kind of experience than if you just read about it and move on.
The New York Times did a great story some years ago on related idea, called Scientific Myths That Are Too Good to Die. It documented how well-known experiments could become sort of “academic urban myths.” Take, for instance, the experiment that lent it’s name to the oft-cited “Hawthorne Effect” (in which the participants’ mere knowledge that they’re part of an experiment skews results):
”The results of this experiment, or rather the human relations interpretation offered by the researchers who summarized the results, soon became gospel for introductory textbooks in both psychology and management science,” said Dr. Lee Ross, a psychology professor at Stanford University.
But only five workers took part in the study, Dr. Ross said, and two were replaced partway through for gross insubordination and low output.
A psychology professor at the University of Michigan, Dr. Richard Nisbett calls the Hawthorne effect ”a glorified anecdote.”
These “glorified anecdotes” (and glorified ballpark guesses, which is really what the percentage-retention statistic is) hang on, though, because, in Dr. Ross’ words again, “’sometimes a story deserves to be true.” That is, the story or number itself may be wrong, but it may be a way to access a point that deserves our attention.
So, then, is a bad statistic in a good cause worthwhile? What if my “90% retention” number gets that grumpy admin to allow my pet wiki project? Is it worth it? I say no, for reasons that lie outside the scope of this post (maybe next one?). Any other opinions, though?
