## Assessment: The Soft Spots

May 29th, 2007 by Dan Meyer

**Introduction**

Jonathan, Jackie, and Sara have been asking sharp questions in the comments about how I assess students. They've found a lot of soft spots on an otherwise leathery-tough assessment strategy and I'd like to address them here.

**Re-Introduction**

First, to bring those up to speed who don't feel like digging through the pdf manifesto (gonna get on that soon, promise):

You break your curriculum into forty skills or concepts.

You take the concepts several-at-a-time as you roll through the year.

You assess a student once with a straightforward problem; that's good for a B.

You assess her again at more depth. Ask her to go backwards and solve for the inputs. Use a word problem. Use negatives. Make her *prove* she's got it.

Then change her B on that concept to an A. Also, tell her she doesn't have to take that concept on future tests.

**Sara's Concern: Retention**

Sara: With shorter teach-practice-assess cycles, how do you know that kids are really learning a concept, not just shoving it into short-term memory for long enough to pass your mini-assessment?

My response from the comments:

Sara, it’s always encouraging and depressing at the same time when people seize on the most obvious glitch in this system. Glad you people are evaluating this thing through critical eyes.

So I’ll say here that once a student completes a concept

twiceand I tell her she doesn’t have to pass it again, that she can work on other concepts, thereisa tendency to file that knowledge away somewhere impermanent.

Butin nearly every case, when I toss an old problem on the board and a student says, I don’t remember that, it takes the absolute minimum of prodding for her to generate full recall.

There's probably a decent discussion to be had here on the merits of retention, in general, in an age when anything can be found on the Internet and everything is kind of like riding a bicycle. At some point in that discussion I'd mention that I had to re-teach myself several sections of Geometry before teaching it for the first time this year. Unfortunately I'm just not courageous enough to make that whole case right now.

**Jonathan & Jackie's Concern: Intellectual Simplicity**

Jackie: Dan, when assessing one concept at a time, how do you assess your students’ ability to synthesize the concepts? Ability to problem solve? Ability to communicate mathematical thinking? In short, when do the higher order skills come into play?

Jonathan: By testing in little pieces, one skill at a time, are they ever asked to put skills together, to use more than one at a time?

There's rigor and there's synthesis. As arbitrated by California's released questions I hit rigor but I rarely *assess* synthesis. Here's the difference. To assess "Similar Area/Volume," recently, I asked the following question:

A large deck weighs 1600 lbs. and costs $40.00 to waterproof. A similarly shaped smaller deck weighs 200 lbs. How much will it cost to waterproof?

By California's standards, that's a rigorous assessment. Jonathan and Jackie are concerned, however, that my concepts don't talk to each other. A common synthesis question (though an admittedly annoying example) would've read:

A large deck weighs 2x – 100 lbs. and costs $40.00 to waterproof. A similarly shaped smaller deck weighs 200 lbs and costs $10 to paint. Solve for x.

To solve the second variety, you've got to know your order of operations, your algebraic equations, *and* your similar area/volume.

I'm pretty sure Jackie and Jonathan understand my intent. I can't crash concepts together like that without gumming up my remediation process. Say a student botched that last question, scored a 2/4. It would then be impossible for me to determine (from the score alone, a week down the line) whether the student understood similarity but just couldn't deal with the algebra or vice versa.

So I split the three components apart and assess them separately.

I'm not sure it'll reassure anyone but me that we *do* hit these synthesized problems hard during openers / classwork / project time. That gets my guilt way down. The hard question here is this: am I willing to fail a student for an inability to synthesize concepts?

My answer is an emphatic no so I keep this game up without much guilt. If it really bothered me, though, I would toss in a concept called "Synthesis" every few concepts. It'd be a lame duck for remediation (though math assessment across the land right now is one loudly-quacking lame duck) but it'd ding students' grades who couldn't synthesize while still leaving me the original, unsynthesized concepts to assist in their remediation.

on 29 May 2007 at 5:01 pm1 RobertDan,

I understand the reasonable concern raised in this thread that concepts and concept-based assessments are too skills based and compartmentalized. However, cleverly constructed concepts can force students to the application level of Bloom’s taxonomy and encourage students to be clever and thoughtful. In fact, I would argue that students with weak basic skills and who would otherwise be considered remedial rely more heavily on analysis and application of a myriad of methods for their success because brute force calculations are like kryptonite for them. They have to think because they have difficulty with the basics and need “easy” paths to success.

For example:

Concept 12: I can solve a quadratic equation using an appropriate method.

a) x^2 + 4x = -3 b) x^2 + = -4x + 1 c) 4x^2 + x = 5

Easy Right? Not if you are a struggling math student. One might suggest that simply memorizing and applying the quadratic formula would be enough for this question thereby making this a simple skills-based question.

Here’s the rub. For a struggling student to use the quadratic formula, they must (without making errors) work with integers flawlessly, simplify radical expressions, and reduce fractions with radical numerators. With each additional step a weak student completes, the likelihood of an error increases.

Therefore, the focus of instruction is necessarily skills-based to some degree in that eventually the students will need to be able to work with the QF efficiently and apply brute force. However, in remedial classes the need for students to analyze different types of quadratic equations and classify them into types most easily addressed by factoring, completing the square, and the quadratic formula is paramount for these students to be successful. Further, these students need the proficiency to apply these skills when needed and without direct prompting to do so. Returning to the problem (Concept 12) above illustrated this point.

FOR A) The weak student should use factoring and the zero product property to avoid the radical simplification and fraction reducing involved in solving this problem using the QF. Factoring allows the student to solve the problem in two steps minimizing the likelihood of errors.

FOR B) The weak student should use completing the square because this seemingly simple problem KILLS students who try to use the QF at the point of radical simplification and again when they have to reduce the resulting fraction containing a radical in the numerator. Completing the square avoids all of these pitfalls because with the even middle term it will have no denominator.

FOR C) They have no choice but to use the QF, but rather than having to muscle their way through three problems they only have to attack one.

The higher levels of Bloom’s taxonomy are necessarily well represented in this schema which also, I believe, improves student retention related to this area of study.

Lastly, a concept based assessment program builds in ever-present moments for student encouragement. You can always find SOMETHING in a test that you can use to say to a kid, “wow, you look like you are really starting to get this,” no matter how poorly he or she is performing overall. Encouraged students are more likely to try more challenging Synthesis-type questions as part of a lesson or as a quiz if they feel more hopeful and empowered overall.

on 29 May 2007 at 9:30 pm2 danYeah, good promotion of retention there and you certainly don’t need to convince me that this assessment strategy can be rigorous.

Reading your comment reminded me also that writing my own assessments, and forcing myself to undergo the same analytical train your comment rides, has done wonders for me as a teacher. I reckon it’s possible to land the balance of instruction/assessment in other ways, this one just feels like an express route.

on 30 May 2007 at 3:34 pm3 JackieDan,

I appreciate the clarification/additional details.

So much to consider as I’m planning for next year… which is a good thing, as I’m actually engaged in thoughtful planning, so thanks!