dy/dan

Who is better at Doodle Jump? Mike or Dan? Why?

The first semester ended, not with a bang, but with two days of canceled class¹ and two days of hasty final exams. My remedial Algebra class spent a lot of time this semester on what California calls computational fluency and what I would rather call the awesome descriptive power of numbers.

Which has meant, thus far, everything from times tables to proportions to infographics all leading to the motivation for the question above: when your friend is being kind of insufferable about how good he is at Doodle Jump, you can use numbers to shut him up!

It is a feature not a bug, in my opinion, that Mike and Dan can draw their own self-serving conclusions from the same set of numbers.

1. … because you can't be too careful with those Santa Cruz tornadoes. [back]

May as well get this out of the way as long as I'm in this public state of contrition.

The concept checklist, in theory, is where students track their progress towards mastery. They write down concept names in rows as we test them and then record their scores (on a four-point scale) along that row, one after the other, each time they retake a concept quiz. I log only their highest scores in the gradebook and whenever they record two perfect scores on a concept, they never have to take that concept again.

The concept checklist is a mess. I run through the same script every year, illustrating the same process with better and more precise visuals every year to no avail. The process confuses students. The process puts students farther from meaningful self-assessment not closer. I saw another checklist crumpled in the trash last week and figured it out.

Their highest score matters much more to me than the specific ordering of low scores preceding it. So forget the earlier low scores. Students add length to the bar as they improve on earlier scores. This checklist design is consistent with our class ethic that "what you know now matters to us more than what you used to know," whereas the other design maintains a permanent record of "what you used to know."

So here's an updated attachment.

BTW: Reader Jacob Morrill does me one (or two or three) better with his adaptation, which is superbly designed:

I'm still compiling my notes from a very strange and very cool CMC-North. Until then, consider this graphic, ripped from children's television by Bill Farren as a visual assessment for engineering students:

I have underrated the assessment question, "what's wrong here?" I need to do more of that. It isn't that tricky, though it is tricky to deliver that assessment visually, as Bill has done here. It's trickier still to rip that visual from a kid's show, packaging the whole assessment in the sort of scientific put-down of children's entertainment that appeals directly to the inner misanthrope I keep loosely tethered on a fraying leash.

Comments are closed here. Tell Bill what's wrong over there.

Jessica, last week, working through a classwork assignment:

Mr. Meyer, where does this go in PowerSchool? Because I check and my grade doesn't change.

Christy, next to her, jumping in:

It doesn't. I checked. But I'm sure he'd take away points if we didn't do it.

Which, um, isn't exactly true.

Perhaps I'll mention some day before the end of the year that none of the classwork they've done all year long has had any direct positive or negative effect on their grade, that the only direct effect of their practice has been on the level of waste material in our recycling bin.

That admission might provoke an interesting conversation about the point of the practice. Or it might provoke riots.

More likely is that I'll chicken out of that conversation until a student distributes printed copies of this blog post to the entire class. That will be fun.

[BTW: It took five weeks, it turns out, for a student to call me out.]

I let Chuck, Shelley, and Robert skip the final exam. We logged fifteen concepts in the first semester of Algebra 1 and those students studied them, practiced them, and demonstrated mastery on all of them. Take a break, kids.

But what if I had given them all fifteen of those concepts again. How accurate is my ranking not just of those three kids but of all of my kids? I have ranked everyone on a four point scale on each of those concepts. Will a student ranked at 2 ("major conceptual errors") again score a 2?

In lieu of a 50 question scantron final, I re-assessed every student on every concept, entered the current ranking into Excel alongside the student's old ranking, and took the difference.

Should've left well enough alone, right?

How Accurate Were The Old Rankings?

• Okay, so big sigh of relief that, in 313 instances, my old ranking was an accurate assessment of a student's current knowledge. Could've been worse.
• Could've been a lot better. That's only 47% accuracy. And in 43 instances, my old ranking was three levels too high. That would be putting a student at a 3 ("minor mechanical errors") and watching the student stare totally blankly at the question on the final forty-three times.

What Does Mastery Mean?

If I have a student ranked at mastery, would she master the same concept on the final exam?

• This isn't awful. This isn't great. I don't know at what point I should be unhappy.

Enduring Questions

• What do we mean when we say "mastery"? Does that mean a student will score perfectly on the same concept every time? Should I be unhappy that the correct/incorrect balance wasn't 100/0?
• What do we mean when we say "retention"? This is a common question of my assessment strategies. "Don't kids forget?" Obviously, I can now answer that question, "yes, sometimes."
• What do we mean when we say "grades"? I don't know what kind of results here would prompt me to pack up the shop and dole out monthly, summative unit exams ("Chapter 6 Test") with the rest of my department. The fact is that this kind of precision analysis isn't even possible under a unit exam model, which puts other teachers in an enviable position; the question "do these assessment scores represent my students' current knowledge?" cannot be answered so it goes unasked. The answer, I'm afraid, is that their assessment scores underestimate student knowledge since Chapter 7 clarified many of Chapter 6′s concepts but these teachers have no mechanism for class-wide re-assessment. So they lower assessment's grade weight beneath that of homework, instead, and inflate their grades with a few extra credit assignments. Look, I'm open to absolutely anything. I just want my grades to mean something. And I need to respect what few guiding principles for assessment make sense to me.