The Tyranny of Statistics

It's problematic enough that people, even experts, misinterpret statistical data (see Kahneman), but statistical data in the absence of a reasonable paradigm just becomes tyrannical.  We pursue or are pursued by data without rhyme or reason.

Matthew Di Carlo:  One can only cringe at the thought of the groups of hard-working teachers in high-poverty schools being shamed annually by a system that dooms them to low ratings by virtue of their dedication to serving the kids who need them the most.

This is why teachers need to articulate what teaching is.  Without a clear and concise description, the bureaucrats, statisticians, politicians and corporations will decide what it is for us.

How Science Can Improve Teaching

Nice article in Scientific American by Daniel T. Willingham.

Down to One Sentence

Teaching is a complex activity, but it isn't beyond comprehension. I have tried to understand teaching well enough to define it in a single sentence. This isn't just an idle puzzle - I think it's important to be able to describe, succinctly yet completely, what one does. The process of boiling it down forces one to focus on essentials, but the danger is in going too far, distilling past the point of usefulness. The definition needs to be an effective tool, not just an evocative vision.

So what is teaching? Well, the teacher does something, and then the student does something and thereby learns. What kind of something? Let's say that the teacher performs a certain action and the student mimics the action. The teacher watches the student, intervenes when necessary, and repeats the action. The student tries again. The process is repeated until the student is capable on his or her own. Teaching by example is probably the most fundamental and natural form of teaching.

But more is going on than meets the eye. A relationship of trust has been built between the teacher and student so that the interaction can work effectively. The teacher's intervention is extraordinarily important, and will depend exactly on what the student has grasped and what he has missed. The teacher directs the student's attention to the work: like this, not like that, here's why, can you see? Mere mimicry is not enough, the student must develop some understanding. The teacher will not always be there, the student must learn to correct himself, must learn how to learn.

In the modern classroom, teachers work with groups of students, sometimes large groups. Directing the attention of a group is an act of performance. For good or ill, the students reinforce each other. Trust is built between teacher and group, usually one student at a time. Each student brings his or her challenges, and instruction must be tailored accordingly.

The instruction is carefully designed so that if the student attends to the instruction, he will learn something. The student may need to have his attention corralled and directed. The teacher may employ a number of behavioral techniques to do this. So we approach a single sentence: Using behavioral techniques, a teacher directs a student's attention to instructional activities designed to help the student learn. Now I have a tool that can help me understand and improve my own work and help others see what it is that I am doing.

Of course, teaching does not happen just in classrooms. It happens all the time, as does learning. Deliberate teaching in any environment can be broadly defined in a similar, more comprehensive, sentence. Teaching is a kind of performance in which the attention of the student is exploited to induce an experience that will change the way the student thinks.

The Halo Effect

The first impression we have of a situation or a person will often color any subsequent judgements we make.  If we like a person, we will tend to like everything about him, even the things we know nothing about - we'll give him the benefit of the doubt.  This tendency is called the halo effect, and it has been studied for many decades.

From Daniel Kahneman, Thinking, Fast and Slow:

Early in my career as a professor, I graded students' essay exams in the conventional way.  I would pick up one test booklet at a time and read all that student's essays in immediate succession, grading them as I went.  I would then compute the total and go on to the next student.  I eventually noticed that my evaluations of the essays in each booklet were strikingly homogeneous.  I began to suspect that my grading exhibited a halo effect, and that the first question I scored had a disproportionate effect on the overall grade.  The mechanism was simple: if I had given a high score to the first essay, I gave the student the benefit of the doubt whenever I encountered a vague or ambiguous statement later on.  This seemed reasonable.  Surely a student who had done so well on the first essay would not make a foolish mistake in the second one!  But there was a serious problem with my way of doing things.  If a student had written two essays, one strong and one weak, I would end up with different final grades depending on which essay I read first . . .

I adopted a new procedure.  Instead of reading the booklets in sequence, I read and scored all the students' answers to the first question, then went on to the next one.  I made sure to write all the scores on the inside back page of the booklet so that I would not be biased (even unconsciously) when I read the second essay.  Soon after switching to the new method, I made a disconcerting observation: my confidence in my grading was now much lower than it had been.  The reason was that I frequently experienced a discomfort that was new to me.  When I was disappointed with a student's second essay and went to the back page of the booklet to enter a poor score, I occasionally discovered that I had given a top grade to the same student's first essay.  I also noticed that I was tempted to reduce the discrepancy by changing the grade that I had not yet written down, and found it hard to follow the simple rule of never yielding to that temptation . . .

I was now less happy with and less confident in my grades than I had been earlier, but I recognized that this was a good sign, an indication that the new procedure was superior.  The consistency I had enjoyed earlier was spurious; it produced a feeling of cognitive ease, and my [judgement] was happy to lazily accept the final grade.

Related websites:
Daniel Kahneman, Thinking, Fast and Slow
The Halo Effect from Wikipedia

Magical Thinking

When I was a boy, I was considered "smart" because I had good grades.  I had no idea why I had good grades, or what "smart" even meant, I was just a boy playing with my friends and going to school because I had to for some reason.  I rarely saw my neighborhood pals in school; I was placed in classes with other "smart" kids and had to wait until the end of the day to play with my friends.

The parents of my friends knew I had good grades, and thus was "smart" (though I doubted they understood the connection any better than I did).  They were happy that their children played with me because this meant that somehow their children might also become "smart."  They even encouraged their kids to stick close to me and watch me because "maybe you'll learn something."  This bothered me tremendously because it not only put my friendships at risk, but it also was patently illogical, even to a boy in elementary school.  "Smart," whatever it was, was not contagious, like a disease.

Now I understand that this was an example of magical thinking.  "Thinking" because it is a kind of thinking, though not logical, and "magical" because belief in the thought is so compelling that evidence or logic to the contrary does nothing to dispel that belief.  It just seems like it ought to be true no matter what.

There is a great deal of magical thinking that goes on in both families and schools.  The example from my boyhood is just one instance of the magical thought that being near something (or looking like or acting like something) will endow you with the qualities of that something.

Here's an example of a kind of magical thinking that happens in schools all the time.  A certain ability or act, like being attentive, often has a certain appearance, like being still and focused.  Taking on that appearance, therefore, will result in that ability.  In other words, if A results in B, then certainly B must result in A.  According to my example, forcing students to be still and to look focused will result in their being attentive.  A variation: students who are still and who look focused ARE attentive, by definition even if not by evidence.  And a corollary: students who are not still and do not look focused can't possibly be attentive to anything.

I am not saying that there is no connection between being attentive and being still.  Certainly both activities often occur together (are highly correlated), and stillness may, under various circumstances, aid attentiveness.  The relationship needs to be clarified, studied, and understood though before we draw the magical conclusion that B must result in A.

I'm sure that my friends' parents saw that I was not a troublemaker, and that I was more likely to correctly weigh the consequences of my actions.  I could be a good example for those of my friends inclined to copy me.  One could conceivably make a connection between these favorable traits and high grades in school.  Somewhere in the mystery of how all this goes is the idea of "smart."  But I guarantee you that the only thing contagious about me were my germs.