Regression to the Mean

Regression to the mean is a little-known and a difficult-to-understand concept. Nevertheless it is quite important. Let me begin with an anecdote from Kahneman’s Thinking Fast and Slow. John Brockman, the editor of the online publication Edge, asked scientists to report their favorite equations. Kahneman’s contributions were as follows:

success = talent + luck

great success = a little more talent + a lot of luck

It’s important to appreciate that there is a component of luck in all success (see the healthymemory blog post, “The Matthew Effect”). Talent and preparation can increase the probability of success, but there is always a component of luck, of being in the right place at the right time. I believe that historically there have been many outstanding people who were never recognized as such. The lack of connections, plus being in the wrong place at the wrong time is the likely reason. Today I believe that there are many more people making notable contributions, or potential contributions, who are never noticed given the vast amount of content in cyberspace.

Regression to the mean was discovered and named late in the nineteenth century by Sir Francis Galton. This was two hundred years after the theory of gravitation and the development of calculus. The definitive article was published in 1886 with the title “Regression towards Mediocrity in Hereditary Stature.” He found that offspring did not tend to resemble their parent in size, but to tend to be more mediocre than their parents. If their parents were large, they tended to be smaller than their parents. If their parents were small, they tended to be larger than their parents. Remember that the mean is average, and the tendency is for extremes to regress to the average.

This phenomenon provides just one of many reasons control groups are needed. Suppose you read an article that reported that depressed children treated with an energy drink improved significantly over a three-month period. Would you accept the conclusion that the energy drink reduced the depression? You shouldn’t unless there was a control group of depressed children who were not given the energy drink. The reason for this is extremes regress to the mean, so you would expect improvement on these statistical grounds alone. So the control group is needed to assess the amount of improvement that can be attributed to statistics alone.

Should you find this concept of regression to the mean difficult, you are not alone. Lawyers hate to make this argument to a jury.

We humans do not like to accept statistical explanations. We like to look for causes. Consider the following true statement:

Highly intelligent women tend to marry men who are less intelligent than they are.

I’m sure you can think of reasons that this might be the case. Moreover, the reasons might be true, but the following statistical result can also account for this relationship, “The correlation between the intelligence scores of spouses is less than perfect.”

© Douglas Griffith and, 2014. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Douglas Griffith and with appropriate and specific direction to the original content.




Tags: , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: