tests

(This is the second of three parts on regression toward the mean.)

If you don’t understand randomness, you can have a hard time telling a real problem from chance fluctuations. This is especially true when it comes to tests of any kind.

Your grade on any test is a combination of luck and skill. I am not going to argue how much of each goes into the mix. Just accept for the moment that to the degree that luck is involved, your test score will vary.

Let’s say you get two A’s in a row. What could explain that?

• You were lucky –that is your true abilities fall below the A range
• You are an A student–that is, your average score is in the A range

Time, and a few more tests,  will tell which answer is correct.  But almost nobody waits around that long.

Apparently the concept of “random variation” didn’t have much survival back when we were evolving those big brains of ours because random chance isn’t the first explanation we come up with. Instead, we’ve evolved to be superb pattern recognizers and inventors. We are emotionally tuned to find patterns and solutions. We comfortable and feel in control when we think we have “explained” things.

“Uses time wisely” and “Works to potential” were my two worst conduct grades in grade school and high school. Lots of A’s, lots of F’s and everything in between.

First semester of my freshman year, I faced two very difficult final exams. Based on past experience I expected to get “bad” grades. Instead, I aced both tests. I cast about for an explanation.  I recalled that I was wearing the same shirt both times.  Maybe the shirt was lucky.  My prefrontal cortex told the rest of my brain that this was superstition. But I could not shake the association. I remember thinking; “This is nonsense. The shirt had nothing to do with it. But what if I am wrong? Who cares, nobody has to know. What harm can it do? “ I wore that shirt to every test, for the next 4 years and couldn’t help myself—I wore to my oral exams for my PhD.

Between High School and College I had become an A student—I’m not exactly sure how. But my first semester I didn’t know that, and so I came to believe that I had a lucky shirt. This particular lack of logic didn’t hurt me much, but similar reasoning can mean unnecessary criticism of the struggling student and it can prevent a mediocre student from taking actions that really will lead to improvement.

Every semester I have conversations like this with at least one of my students:

Student: Dr. Argiros, I failed Sociology this month what should I do?

Me: What did you get?

Student: 71

Me: What grade did you get last month?

Student: 77

Me: And the month before?

Student: 75

Me: So, you have been passing so far, but just barely, and this month your grade is a little below the bar. Is that right?

Student: Yea, I’m not sure what happened.

Me: The difference is just a few points; do you think it might have been bad luck?

Student: Surprised at my response, Maybe, but I really didn’t understand all that stuff about capitalism we had to learn this month. I liked the stuff we were doing before that better.

Me: O.K. I can buy that. It’s easier to learn something if you like it. But I still don’t see a big difference between a 71 and a 77. I don’t think you should focus on what happened this month only. Whatever you are doing now is good enough to get you grades in the 70s. If you don’t want to get a grade lower than a 75 then you need to figure out what to do so that your average scores are higher—in the 80’s. That way if you are unlucky one month you will probably still pass. Does that make sense? If you like, I’d be happy to look at your notes and we could talk about what you are focusing on …

Sometimes this works. But more often than not, the student never comes to talk to me about study skills.

Either in the conversation with me, or later on when he is discussing the situation with parents and friends, almost invariably the focus shifts back to a comparison between this month’s failing grade and last month’s passing one. Having decided that he only passes when he is interested. He may decide there is nothing he can do except hope that he likes the rest of the semester better.

Even if he decides he wants to do better, unless he changes the way he has framed the problem, whatever he picks as the “reason” is not likely to do much good. It will have little more real impact than my “lucky” shirt. Here is the kicker. Because of regression toward the mean, there is a very good chance that next month’s test scores will be higher anyway. This will confirm the change in his mind. And he will continue another month and another—at least until his luck changes. Score: Superstition:1 Improvement:0.

Tagged as: chance, grades, prefrontal cortex. evolution, randomness, Regression toward the mean, supersition, tests

{ 4 comments }