Posts

Comments

Get Posts by E-mail

Archive for January, 2009

I let Chuck, Shelley, and Robert skip the final exam. We logged fifteen concepts in the first semester of Algebra 1 and those students studied them, practiced them, and demonstrated mastery on all of them. Take a break, kids.

But what if I had given them all fifteen of those concepts again. How accurate is my ranking not just of those three kids but of all of my kids? I have ranked everyone on a four point scale on each of those concepts. Will a student ranked at 2 ("major conceptual errors") again score a 2?

In lieu of a 50 question scantron final, I re-assessed every student on every concept, entered the current ranking into Excel alongside the student's old ranking, and took the difference.

Should've left well enough alone, right?

How Accurate Were The Old Rankings?

  • Okay, so big sigh of relief that, in 313 instances, my old ranking was an accurate assessment of a student's current knowledge. Could've been worse.
  • Could've been a lot better. That's only 47% accuracy. And in 43 instances, my old ranking was three levels too high. That would be putting a student at a 3 ("minor mechanical errors") and watching the student stare totally blankly at the question on the final forty-three times.

What Does Mastery Mean?

If I have a student ranked at mastery, would she master the same concept on the final exam?

  • This isn't awful. This isn't great. I don't know at what point I should be unhappy.

Enduring Questions

  • What do we mean when we say "mastery"? Does that mean a student will score perfectly on the same concept every time? Should I be unhappy that the correct/incorrect balance wasn't 100/0?
  • What do we mean when we say "retention"? This is a common question of my assessment strategies. "Don't kids forget?" Obviously, I can now answer that question, "yes, sometimes."
  • What do we mean when we say "grades"? I don't know what kind of results here would prompt me to pack up the shop and dole out monthly, summative unit exams ("Chapter 6 Test") with the rest of my department. The fact is that this kind of precision analysis isn't even possible under a unit exam model, which puts other teachers in an enviable position; the question "do these assessment scores represent my students' current knowledge?" cannot be answered so it goes unasked. The answer, I'm afraid, is that their assessment scores underestimate student knowledge since Chapter 7 clarified many of Chapter 6′s concepts but these teachers have no mechanism for class-wide re-assessment. So they lower assessment's grade weight beneath that of homework, instead, and inflate their grades with a few extra credit assignments. Look, I'm open to absolutely anything. I just want my grades to mean something. And I need to respect what few guiding principles for assessment make sense to me.

I give my students two problems a night — challenging, standards-based, assessment-grade cuts off Algebra's prime rib. They can choose from the easier or harder problem. It drives me nuts that some students will still blow off such a low-volume assignment.

I was threatening my Algebra 1 sections with an increased problem count, increased reward for completion, and increased penalties for non-completion as of the second semester, which started today. Then I graphed their first semester grades against homework completion and gave up.

Then you have my three sections of remedial Algebra, which behave more or less as you'd expect.

It's all very troubling.

We have entries from Collette Cassinelli, Simon Job, Fred Knauss, Alice Mercer, and Sam Shah (did I miss a trackback?) with five days remaining in competition1. I finished my own (out of competition) entry last night while recovering from a flu / bronchitis combo and posted it here.

I abandoned it, to a certain extent. My content means a lot to me, if only because I bothered to track the data all year long, but my design work is simply functional — a staid set of bar charts and line graphs. See you next time.


  1. Entry page here.

started my dy/dan 4 slide project. i might not finish… just not inspired and don't have knowledge of the tools to do this. — Sam Shah, who finished second place (!) last year.

I was shocked, frankly, we had twenty entries last year to what was a pretty demanding competition. I'm pushing my luck a second time only because I'm enjoying a fairly transcendent experience designing my own, the kind of happy nerdery you can't keep to yourself, you know?

I mean, this bar chart is sharp as a blade, right? Any guesses what month my long-distance girlfriend became my close-distance wife?

That's one of, like, sixteen visualizations I'm working on. We're looking for four. Possible data sources:

And, for the love, do not neglect PivotTables, your best offense against huge piles of data.

I'm positive he faked this, though. That right there looks better than my right-side-up penmanship.

Next »