Brett Gilland coined the term “mathematical zombies” in a comment on this blog:

Students who can reproduce all the steps of a problem while failing to evidence any understanding of why or how their procedures work.

When I think about mathematical zombies, I think about z-scores — how easy it is to *calculate* them relative to how difficult it is to *explain* those calculations.

Check it out. Here is the formula for a z-score:

In words:

1. You subtract the mean from your sample.

2. You divide that by the standard deviation.

Subtraction and division. Operations simple enough for a elementary schooler. But the *explanation* of those operations —Â why they result in a z-score, what a z-score is, and when you should *use* a z-score — is so challenging it eludes many graduates of high school statistics. Think about how easily you could solve these exercises without knowing what you’re doing.

That difference brings this chart to mind and helps me understand all of the times I’m tempted to just tell students, *here’s how you do it already so now just do it*. That’s where the operational shortcuts are most tempting.

All of this is preface to a lesson plan on hypothesis testing by Jeremy Strayer and Amber Matuszewski, which is one of the best I’ve read all year.

Hypothesis testing is, again, one of those skills that’s far easier to *do* than to *understand*. As you read the lesson plan, please keep in mind that difference. Also notice how capably the teachers develop the question, disclosing the mathematics progressively, and resisting the temptation to shortcut their way to operational fluency.

It’s spectacular. I’m struck every time by a moment where Strayer and Matuszewski ask students to model an experiment with playing cards, only to model the exact same experiment with a computer later. They didn’t just jump straight to the computer simulation!

Here is a video of an airline pilot landing an Airbus A380 in a crosswind. This is that for teachers.

**Featured Comment**

Amy:

I always think of z-scores as a set of transformations from one plain-vanilla normal curve to the hot-fudge-sundae Standard Normal curve. Maybe once you see it this way, you can’t unsee it. To me, that helps make sense of the “why” you would bother standardizing and the “how” it’s done.

I’m not sure I agree that z-score is so conceptually difficult as to be worth the shortcut. Though I suppose it requires understanding of standard deviation, which is kind of hard. But if you think of standard deviation as “typical weirdness distance” then z-score as the idea of “how many times the typical weirdness is this point” becomes pretty straightforward. A z-score magnitude of 1 becomes average weirdness, less than 1 becomes less weird than average, etc. The bigger the magnitude of the z-score, the weirder the point.

In introductory stats courses, much of what we do simply comes down to separating “Is it possible?” from “Is it plausible?”. We have seen a wonderful growth in the number of free, online applets which allow teachers and students to perform simulations designed to assess these subtly different questions.

## 38 Comments

## Andy

September 8, 2016 - 10:40 am -John Hattie has been roasted for his misuse of z-scores in his Visible Thinking research. Even well-known scholars have mathematical zombie moments!

## Howard Phillips

September 8, 2016 - 12:07 pm -And so it should be done this way, at least a few times.

Some thought should be given to the standard deviation as well, since the mean absolute deviation is what is used up to grade 8, and that standard deviation gets one mention in high school.

## David Griswold

September 8, 2016 - 12:12 pm -This is a great lesson! Definitely a great way to introduce hypothesis tests (or simply sampling distributions.) Just wrote most of a similar lesson myself, but this one is more thought out. May need to borrow the use of cards…

I’m not sure I agree that z-score is so conceptually difficult as to be worth the shortcut. Though I suppose it requires understanding of standard deviation, which is kind of hard. But if you think of sd as “typical weirdness distance” then z-score as the idea of “how many times the typical weirdness is this point” becomes pretty straightforward. A z-score magnitude of 1 becomes average weirdness , less than 1 becomes less weird than average, etc. The bigger the magnitude of the z score, the weirder the point.

## Bob Lochel

September 8, 2016 - 2:03 pm -In introductory stats courses, much of what we do simply comes down to separating “Is it possible?” from “Is it plausible?”. We have seen a wonderful growth in the number of free, online applets which allow teachers and students to perform simulations designed to assess these subtly different questions. Allan Rossman and Beth Chance have continued to be on the front lines of providing wonderful resources, and the Lock5 family are also inspirations.

But no matter how much the technology improves, nothing causes immersion in context and meaningful conversation than tactile simulations. The dolphin example is excellent, and I’ll also point out that many AP Stats teachers are buzzing this year over Doug Tyson’s powerful Smelling Parkinson’s activity: http://mrtysonstats.com/apstats/apstats.html, and Distracted Driving from NCSSM also using playing cards effectively: http://courses.ncssm.edu/math/Stat_Inst/Stats2007/Distracted%20Driver/Distracted%20driving%20Teacher%20version%20final.pdf.

## Jerzy

September 8, 2016 - 3:50 pm -I love most things about this lesson, and I plan to use it in my own teaching.

But I would redirect the discussion at the end. They mention students’ concerns about sample size, which could have led to a great discussion about power and effect size. The real point of hypothesis tests isn’t to answer

“Is the true effect exactly 0 or not?” (it rarely is!)

but rather

“Did we measure the effect precisely enough to be confident that it’s positive or confident that it’s negative?”

When you fail to reject the null, you’re not concluding the effect is 0; you’re just concluding the effect wasn’t measured very precisely. I’d love to find good lesson plans / resources for getting this idea across.

And I can’t help wishing that both the teacher and students had discussed the p-value correctly. For example:

“There appears to be only a 1.2% chance that randomness alone accounts for the difference of 7 that the researchers found in the Dolphins study or a value more extreme than 7.”

This should be something like:

“If randomness alone accounted for the difference that the researchers found in the Dolphins study, there appears to be only a 1.2% chance that it would have been a difference of 7 or more extreme.”

And later:

“Believing the null hypothesis is true, the probability of the results being random are extremely low.”

could be simply

“Believing the null hypothesis is true, the probability of the results is extremely low.”

Again, I haven’t found a good way to get this idea ingrained in students.

Still, thanks for linking to this great resource.

## Howard Phillips

September 8, 2016 - 4:36 pm -As we have considered the z-score as a continuous variable the correct z-value should be (6.5 – mean)/std. dev.

## Greg Ashman

September 8, 2016 - 7:25 pm -Hi Dan

I agree with you totally about z-scores – easy to calculate and difficult to explain what’s going on. I wonder whether you have considered the possibility of showing students how to calculate z-scores *and also* explaining to them what they are etc.? Are these two kinds of exposition mutually exclusive?

Thanks

Greg

## Michael Paul Goldenberg

September 9, 2016 - 8:14 am -You can start further back with a relevant formula for the z-score calculation: where does sigma come from? Again, relatively easy to calculate standard deviations, but not so simple to understand the WHY? of the formula. Frankly, I got A’s in one undergraduate and two graduate statistics courses back in my pre-math teacher days (late 1970s) without seeing the obvious: that the formula for standard deviation is an AVERAGE of deviations from the mean. It’s “so obvious” even from the name, but I was doing mathematics as I’d been taught in K-12 – completely mechanically and procedurally, and since I could solve the problems and get A’s in my classes, who needed conceptual understanding, right?

## Robert

September 9, 2016 - 9:43 am -I would say that that’s a fundamental problem of teaching math at a K-12 level. You’re dealing with an audience that aren’t exactly trained/prepared for mathematical deductions in the first place, so finding a shortcut, dumbing down the material, can sometimes make them pass through the system better – Just not the best long-term solution. Focus on understanding has to be implemented right from the very start of math education.

## Michael Ruppel

September 9, 2016 - 10:23 am -FWIW, given a mean and standard deviation, the question “how many standard deviations away is this from the mean?” followed by “show the calculation that led to this answer” usually gets right answers and a good understanding of the concept from my students. We don’t touch that “formula” until they have it themselves.

## Dan Meyer

September 9, 2016 - 1:17 pm -David Griswold:Thanks for “typical weirdness distance.” Added to the body of the post.

Bob Lochel:Thanks for that distinction. Also added to the post. I’d love some elaboration on how different online applets help students appreciate those distinctions. (In case you’re looking for some fodder for your own blog.)

@

Jerzy, useful clarifications. I passed them along to one of the co-authors.Greg:Certainly. Just to clarify: the “explanation” in this post doesn’t refer to teacher explanation but rather student explanation. Neither this post nor the article (as far as I can tell) argue that teacher explanation precludes student understanding. Both my project and (it seems) theirs is to find effective conditions for teacher explanation.

## Chester Draws

September 9, 2016 - 4:16 pm -The z-score literally

isthe formula’s result. The z-score doesn’t “mean” anything important other than its relevance to a standard Normal curve. And that Normal curve just is what it is — what “meaning” can you ascribe to an idealised curve?So I teach z-scores the same way I teach using medians and means. We have a tool, which we calculate in this manner, and which we use to solve certain problems.

Does anyone teach median’s “meaning” for several lessons before they show how one is found? That would be kind of weird.

I suspect that some students who are called “zombies” are merely those that have an internalised Maths understanding. That they can’t put in words why they do something doesn’t mean that they are doing things without thinking. As you get good at something it tends to become more automatic, and it seems counter-productive to stigmatise those students who are most automatic, i.e. best, at something.

## Dan Meyer

September 9, 2016 - 6:15 pm -I hope you reconsider all of that, Chester.

## Michael Paul Goldenberg

September 9, 2016 - 6:19 pm -@Chester Draws: “I suspect that some students who are called “zombies” are merely those that have an internalised Maths understanding. That they can’t put in words why they do something doesn’t mean that they are doing things without thinking. As you get good at something it tends to become more automatic, and it seems counter-productive to stigmatise those students who are most automatic, i.e. best, at something.”

Sorry, Chester, but no sale. Teachers who promote understanding give students many opportunities and modalities for showing that they have more going on than some sort of automaticity in reproducing steps. There’s a movie made by the U of Michigan mathematics department in which students who made A’s in their freshman calculus classes to explain what the derivative of a function was. Hardly any of them could do more than give an example of how to compute the derivative of some simple polynomial or recite the rules for doing such calculations. Stating that it was a way to find the slope of the tangent line to a function at a given input value of that function or anything vaguely like that, or anything to do with limits, or anything that suggested that they knew why one would ever want to know the derivative of a function? Not on your life.

Those were the very good students who learned math mechanically in K-12 and were continuing to perform trained dog tricks at a very competitive public university. But they weren’t the kids in honor calculus (i.e., introductory analysis) who were expected to be able to do proofs and talk intelligently about what in hell calculus was about. I’ll grant that many mathematicians at U of M I’ve known have said that it wasn’t until grad school or post doc work when they had to teach calculus that they felt that they really understood it, but I don’t think we’re talking about that level of conceptual understanding. Being a facile human calculator is just not all that impressive, truth be told.

And we teach countless students at least through upper division mathematics classes at decent post-secondary schools a bunch o’ calculation routines that they can memorize and use and regurgitate for exams with minimal understanding. What is the meaning of the determinant of a matrix? Do you believe it has any meaning? Or is it simply the result of a series of procedures that we can teach half-way intelligent monkeys to perform? How about the dot product or cross product?

If you think these things are simply the results of calculations that have no meaning, I recommend a current series of videos about the essence of linear algebra available free on YouTube by 3blue1brown. Or maybe not. Maybe you should continue to believe that some of us are making mountains of molehills and that we just don’t appreciate the kid who can number crunch. You’ll certainly be in good company, but it’s not the company I prefer.

## Jeremy Strayer

September 9, 2016 - 8:49 pm -Re: Jerzy on September 8, 2016 at 3:50 pm

Thanks for the comments Jerzy. While our lesson was focused on “developing the question,” as Dan put it so well, we did end a section of the article with a quote from a student who was expressing their understanding as they began to connect the exploration with the more formal language of statistics (p-value in particular). I wish the word limit of Mathematics Teacher (MT) would have allowed us to give a fuller discussion of the nuances of the student’s developing understanding. Also, I wish you would have been a reviewer on the paper so we could have tightened up our own language! Though we did say “or a value more extreme than 7,” your phrasing was tighter. If you don’t already review for MT, you should!

Finally, if you have ideas on a lesson for effect size for high schoolers, get in touch with me. We can write the lesson, try it with students, and write the next MT article!

## Paul Carson

September 10, 2016 - 12:16 am -@Michael Paul Goldenberg. Great comment. That is an idea that I’ve been working on for 16 years and yet it still happens. This year with new students (I’m a private tutor who usually gets students who have just failed or performed poorly) who have got an A up to now in their lives but suddenly get Ds/Es. The reason? They know what to do, but they don’t know what they’re doing. My life mission is to change that.

The mathematical zombies definition has a certain implication it’s the students’ fault – but I think it’s entirely schools’. And it is so sad. Maths is great when you understand it. When a student is asked ‘what is differentiation’ and they say ‘er, you bring the power down and take one off, but I don’t know why’ I feel very angry on their behalf. As Dan says, they’ve been shown the shortcut but that should come AFTER understanding, and even where it comes from. Do the teachers know where that shortcut comes from?

## Bill Shillito

September 10, 2016 - 3:53 pm -“There’s a movie made by the U of Michigan mathematics department in which students who made A’s in their freshman calculus classes to explain what the derivative of a function was.”

Where can I find this video?

## Michael Paul Goldenberg

September 11, 2016 - 8:15 am -Ask the math department. No idea if it is available to people not involved with the study.

## Chester Draws

September 11, 2016 - 1:53 am -Ah, a movie Paul. Well that proves it then!

You can prove almost anything about students if you set up the test to do so, as we both know.

If a student knows that the derivative finds the gradient at a point, how can they not understand the purpose of the derivative?

That it represents the gradient of the tangent at a point is the sort of thing that Maths freaks think is important, but actually isn’t relevant to anything much. Ask them to find the tangent and they can, so they “know” the relationship, even if they can’t put it into the words that Maths professors prefer.

That the derivative is technically found with limits is really not at all important to actually using it. It’s like those people that think that you don’t know what an engine does because you can’t tell them what fuel injection does.

What is the meaning of the determinant of a matrix? Do you believe it has any meaning?It depends what you mean by “meaning”. It has a purpose, a use, which depends on the situation you might be in. I don’t believe it has much meaning outside of a process. So I would teach it in context of that purpose. I wouldn’t spend time explaining the “meaning” of discriminant outside that purpose.

## Chester Draws

September 11, 2016 - 1:54 am -Ah, a movie Paul. Well that proves it then!

You can prove almost anything about students if you set up the test to do so, as we both know.

If a student knows that the derivative finds the gradient at a point, how can they not understand the purpose of the derivative?

That it represents the gradient of the tangent at a point is the sort of thing that Maths freaks think is important, but actually isn’t relevant to anything much. Ask them to find the tangent and they can, so they “know” the relationship, even if they can’t put it into the words that Maths professors prefer.

That the derivative is technically found with limits is really not at all important to actually using it. It’s like those people that think that you don’t know what an engine does because you can’t tell them how fuel injection works.

What is the meaning of the determinant of a matrix? Do you believe it has any meaning?It depends what you mean by “meaning”. It has a purpose, a use, which depends on the situation you might be in. I don’t believe it has much meaning outside of a process. So I would teach it in context of that purpose. I wouldn’t spend time explaining the “meaning” of discriminant outside that purpose.

## Michael Paul Goldenberg

September 11, 2016 - 2:16 am -How very sad, “Chester.”

## Bill Shillito

September 11, 2016 - 5:27 am -Those who say that something like “the determinant of a matrix has no meaning but rather just a purpose” are usually those who just plain don’t know what that meaning is.

The meaning of the determinant of an nxn matrix is that it is the scale factor applied to the n-dimensional volume of an object when it is multiplied by that matrix as a linear transformation. If the determinant of a 2×2 matrix is -6, the associated transformation will multiply the area of a figure by 6 as well as reverse its clockwise/counterclockwise orientation. This also explains why a determinant of 0 means the matrix is non-invertible — if you multiply the volume by 0, there’s no going back and recovering your original volume because you’ve collapsed your figure down to fewer dimensions.

Makes me so sad whenever I open a textbook and all I see about the determinant is that it’s “a number -associated- with a matrix.” Whoop-dee-doo — WHY would I bother associating that number with the matrix? Why couldn’t I have just used ad + bc instead of ad – bc? Probably because the ad – bc actually MEANS something.

## Dan Meyer

September 11, 2016 - 7:52 am -This particular conversation with Chester has run aground on firmly held ideological differences that I don’t think it does us any more good to explore. (Which is to say, I’ll be deleting further comments on the matter.) Any other comments on the post or the article I linked, have at it.

## Jerzy

September 12, 2016 - 11:57 am -@Jeremy – thanks! It really is a great lesson, and I’m passing it along to colleagues revising our (undergraduate) intro stats curriculum. We’re good at nitpicking p-value definitions here :) but I admit that this carries the risk of creating more zombies ourselves. It’s much harder to “develop the question” well as you did.

And I totally empathize with the word limits problem.

I’ll certainly get in touch if I think of an effect-size lesson idea.

## Amy Hogan

September 12, 2016 - 12:13 pm -I always think of z-scores as a set of transformations from one plain-vanilla normal curve to the hot-fudge-sundae Standard Normal curve. Maybe once you see it this way, you can’t unsee it. To me, that helps make sense of the “why” you would bother standardizing and the “how” it’s done.

## Dan Meyer

September 12, 2016 - 12:20 pm -Amy:Hot Fudge Sundae Standard Normal Curve.

Between that and

David’s“Typical Weirdness Distance,” this thread has been rich in explanatory metaphor. Added to the post.## David Griswold

September 12, 2016 - 12:39 pm -Amy, I love the transformation view within the context of the normal distribution, but by itself I always feel it kind of misses what for me is the key point that z-scores (or rather standardized values) do have a meaning outside of the normal distribution. You can standardise the values of any distribution and the standardized value always has a contextual meaning. An even better value would perhaps be (x-median)/MAD, to get at the same idea in asymmetric distributions without as much volatility, but sadly we don’t have a catchy name for that measure.

## Amy Hogan

September 12, 2016 - 1:35 pm -Agreed. Even outside the context of the normal distribution, I think of it as a transformation. In that sense, it tells how unusual a value is in relation to the data’s variability. So we must dilate to consider the variability (dividing by the SD) and translate in the direction and magnitude of the weirdness (by subtracting the mean).

## Pam Rawson

September 12, 2016 - 2:00 pm -As a college student, I was a mathematical zombie – I knew how to calculate and use a z-score and would do so correctly just about every time.

In my teacher life before Baxter Academy I taught with Core-Plus Mathematics, an integrated program with an extremely strong and well-developed statistics & probability strand. It was only while learning with Core-Plus that I truly understood the meaning of a z-score. When I observed students learning through these materials, I was astounded that the approach helped students to develop their own formula by asking the simple contextual question, “How many standard deviations away from the mean …”

A more interesting question for me, however, is why standardize in the first place? We have technology that will give us the p-value without standardizing, so why even bother? I wrote about the way this question played out in the classroom in April 2015. (https://rawsonmath.wordpress.com/2015/04/)

## Paul

September 12, 2016 - 6:59 pm -I currently am a college student majoring in math (minoring in secondary education) and I frequently feel like a mathematical zombie. So much that is being taught is over my head, and professors don’t stop to clarify much of anything. When they do ask for questions, I am so far lost that I don’t even know what to ask.

I feel as though this is similar to how a lot of students feel when asked to explain the concept of something like a z-score or derivative. Since it is so easy for teachers to move past the explanation quickly, as it makes sense to us, students feel lost very easily and are often embarrassed to ask for help. This then snowballs as the teaching continues and they feel even more lost, so they just check out and memorize how to solve the problem. It’s a very unfortunate issue that is also very common.

## Michael Paul Goldenberg

September 12, 2016 - 8:47 pm -@Paul, your experience is extremely common. What’s good is that you’re curious about understanding concepts and aware to some extent that the style of teaching you’re encountering has something to do with what you’re experiencing. Don’t let anyone tell you that it’s your fault (even if you feel that it is to some extent).

There are increasing numbers of marvelous videos available online, generally free, that give treatments of various college math subjects that are more insightful and user-friendly. And even if they weren’t, if the lectures are sound, you can watch them multiple times for clarification. Sometimes looking at a couple of lectures from different professors/teachers on the same topic will do the trick. And there are lots of places to post questions (Quora has some very good math people answering inquiries, for example). Don’t despair.

## Jonathan Osters

September 14, 2016 - 6:28 am -As for z-scores, I always get to the formula THROUGH asking for meaning.

Example: you have a set of data, say goldfish weights. Some numbers large, some small.

Q1: What’s “typical” here? (Review measures of center – kids mostly land on mean being “typical”)

Q2: If I wanted to show how typical or atypical these numbers were, how could I do that? (Many students suggest finding how far each number is from the mean, by subtracting the mean from each number)

Q2a: What do positive/negative/large/small numbers MEAN in this data set?

Q3: Suppose I had another data set that had different data (say, the weights of polar bears), but I wanted to make an apples-to-apples comparison between, say, the heaviest goldfish and the heaviest polar bear, how could I do that? (It’s at this point that discussion relating to variability – how many standard deviations from the mean is it? – leads to the formula for z organically)

For normal distribution, I like Amy’s idea.

Very few of my students ever forget WHY they’re doing the calculations they’re doing.

## Richard

September 14, 2016 - 12:31 pm -Throughout my entire academic life I felt like I was a mathematical zombie. The funny thing is that I was often content with just knowing the operation. I never really questioned the “why” and the “how,” or rather the thought of it never popped up. I thought knowing and memorizing the operation IS the math. It wasn’t until sometime in college that I found out that math was beyond that, but it’s sad that I had to wait till college to find that out.

## Michael Paul Goldenberg

September 14, 2016 - 1:23 pm -It grieves me to say, #Richard, that your experience is typical except that you actually studied math in college at least enough to see a bigger, deeper, more beautiful picture. Relatively few people ever get that far. I’ve had many difficult conversations with students, teachers, parents, and STEM people (who bloody well should know better) on the question of whether “knowing and memorizing the operation IS the math.” It’s not generally enjoyable and only rarely is it productive. But I keep at it because even reaching a handful of people is worth the pain and struggle. And I’d say that’s a lot like studying a challenging subject like mathematics: small successes are worth the pain and struggle.

## Justin Dishong

September 14, 2016 - 7:52 pm -I actually became a math teacher as a “mathematical zombie” and was fully convinced that I would be great at making more mathematical zombies. In the process of teaching the content I began to discover the meanings behind many short-cuts that I had memorized throughout the years, and to my surprise, was enjoying it quite a bit. I started sharing this with my students and was pleasantly surprised when they also found the answer to the “why” questions interesting. I’m still a very young teacher and I have been looking around for resources and/or a community that would help me to pursue and grow in this style of teaching. I just happened to come across your blog tonight and read this post. I teach middle school math, so this was a little out of my league, but I’m excited to check back regularly and read more. Thanks so much for everyone’s insight, expertise, and heart for teaching kids to think deeply.

## Mathew

September 14, 2016 - 9:19 pm -I am a math major in college studying education, and I guess I would say that mathematical zombies are common in the world, but I definitely feel as though I have had a different experience than some. In my high school calculus class my teacher would often prove to us why certain things were true (very informal proofs) and explain what certain things mean. She wouldn’t just let us know how to calculate something, she made sure we could understand it better. Also I think that in college I have experienced a great deal of revelation of truth in my upper division classes.

Also as a statistics fanatic, I agree that z-scores can be a very confusing idea, but when it comes down to it, it is only confusing because of the Standard Deviation. Something few people can probably explain correctly. I think that if you can explain to a student what the standard deviation means well enough, then it becomes natural to try and measure how many deviations away from the average a value is.

## Dan Meyer

September 15, 2016 - 7:39 am -Justin:That’s one of the joys of teaching they don’t tell you about right there. The common understanding of math teaching is that once you max out your understanding of secondary math, you teach from your own frozen, calcified body of knowledge. Not so in your case. Not so in my case. So nice to have a job that offers so many opportunities for continued growth.

## Andrew

September 19, 2016 - 2:06 pm -I’m a college student at the University of Illinois at Urbana-Champaign studying mathematics witha minor in secondary education, and I totally relate to the term of a mathematical zombie. I feel like throughout college, I’ve lost any purpose of taking advanced math classes while trying to become a high school math teacher because frankly speaking, I won’t teach a high school student what these concepts mean.

As a result, I’m doing my best to understand the concepts enough to pass each class and not worry about it too much in the future. If you asked me to tell you that a number is prime, I would be able to tell you how to get there, but I would have no reason why it happens. I’m trying to get away from that mindset, and learn what it means to take that extra step and understand the why behind the what so that I can have the same attitude with my students.