Jonathan, Jackie, and Sara have been asking sharp questions in the comments about how I assess students. They've found a lot of soft spots on an otherwise leathery-tough assessment strategy and I'd like to address them here.
First, to bring those up to speed who don't feel like digging through the pdf manifesto (gonna get on that soon, promise):
You break your curriculum into forty skills or concepts.
You take the concepts several-at-a-time as you roll through the year.
You assess a student once with a straightforward problem; that's good for a B.
You assess her again at more depth. Ask her to go backwards and solve for the inputs. Use a word problem. Use negatives. Make her prove she's got it.
Then change her B on that concept to an A. Also, tell her she doesn't have to take that concept on future tests.
Sara's Concern: Retention
Sara: With shorter teach-practice-assess cycles, how do you know that kids are really learning a concept, not just shoving it into short-term memory for long enough to pass your mini-assessment?
My response from the comments:
Sara, it’s always encouraging and depressing at the same time when people seize on the most obvious glitch in this system. Glad you people are evaluating this thing through critical eyes.
So I’ll say here that once a student completes a concept twice and I tell her she doesn’t have to pass it again, that she can work on other concepts, there is a tendency to file that knowledge away somewhere impermanent.
But in nearly every case, when I toss an old problem on the board and a student says, I don’t remember that, it takes the absolute minimum of prodding for her to generate full recall.
There's probably a decent discussion to be had here on the merits of retention, in general, in an age when anything can be found on the Internet and everything is kind of like riding a bicycle. At some point in that discussion I'd mention that I had to re-teach myself several sections of Geometry before teaching it for the first time this year. Unfortunately I'm just not courageous enough to make that whole case right now.
Jonathan & Jackie's Concern: Intellectual Simplicity
Jackie: Dan, when assessing one concept at a time, how do you assess your students’ ability to synthesize the concepts? Ability to problem solve? Ability to communicate mathematical thinking? In short, when do the higher order skills come into play?
Jonathan: By testing in little pieces, one skill at a time, are they ever asked to put skills together, to use more than one at a time?
There's rigor and there's synthesis. As arbitrated by California's released questions I hit rigor but I rarely assess synthesis. Here's the difference. To assess "Similar Area/Volume," recently, I asked the following question:
A large deck weighs 1600 lbs. and costs $40.00 to waterproof. A similarly shaped smaller deck weighs 200 lbs. How much will it cost to waterproof?
By California's standards, that's a rigorous assessment. Jonathan and Jackie are concerned, however, that my concepts don't talk to each other. A common synthesis question (though an admittedly annoying example) would've read:
A large deck weighs 2x – 100 lbs. and costs $40.00 to waterproof. A similarly shaped smaller deck weighs 200 lbs and costs $10 to paint. Solve for x.
To solve the second variety, you've got to know your order of operations, your algebraic equations, and your similar area/volume.
I'm pretty sure Jackie and Jonathan understand my intent. I can't crash concepts together like that without gumming up my remediation process. Say a student botched that last question, scored a 2/4. It would then be impossible for me to determine (from the score alone, a week down the line) whether the student understood similarity but just couldn't deal with the algebra or vice versa.
So I split the three components apart and assess them separately.
I'm not sure it'll reassure anyone but me that we do hit these synthesized problems hard during openers / classwork / project time. That gets my guilt way down. The hard question here is this: am I willing to fail a student for an inability to synthesize concepts?
My answer is an emphatic no so I keep this game up without much guilt. If it really bothered me, though, I would toss in a concept called "Synthesis" every few concepts. It'd be a lame duck for remediation (though math assessment across the land right now is one loudly-quacking lame duck) but it'd ding students' grades who couldn't synthesize while still leaving me the original, unsynthesized concepts to assist in their remediation.