## On Nailing/Blowing Assessment

May 13th, 2008 by Dan Meyer

**Blowing It**

Me, on our last concept quiz, balling both Law of Sines and Law of Cosines into the same heading:

I watched kids tear Law of Sines apart and then get torn apart by Law of Cosines. I was about to toss 2 points out of a possible 4 into the gradebook for, like, seventy students.

But then they come in for help a week, maybe two weeks down the line and what? How does that 2/4 direct my remediation? Which don’t they understand? Law of Sines or Cosines?

And here I try so hard to imagine: how in the world did I ever lump a dozen skills under the same “Chapter [x] Test” heading, the preferred grading strategy of the world’s math teachers?

Disaggregation is the name of the game. It empowers students and teachers. So, on the next test, I did:

**Nailing It**

Frank N., from the comments, co-opting this assessment strategy for physics.

Now, has all this craziness made a difference? I can tell you this: the kids don’t feel defeated by physics as they did in years past. They can get a 2/10, realize that they didn’t know what they thought they knew, and come back to get a 9/10 and feel great. Plus, when it comes down to grades, there isn’t anything stopping them from getting a 100 each quarter. The ball in in THEIR court. How can a parent argue with a system like that?

In addition, I can immediately tell which topics need re-teaching by me and which the kids get right away.

Exactly.

on 13 May 2008 at 6:03 pm1 Vincent BaxterA quantitative research methods professor used a Chinese proverb to explain to me his thinking on designing assessment. It goes something like this: You find yourself on the bank of a river, wanting to get to the other side. You ask a local fisherman, “Can I walk across? How deep is the river?” He tells you, “On average, it’s one foot deep.” Well, do you trust that? What if you start to walk across and it’s a few inches deep, and then you fall into a chasm? I try to think about testing in the same way: if the class averages 80% on an assessment, they’re passing…but what is in that 20% of misunderstanding? And, is that misunderstanding the same for each kid? No way! What kind of a shift in assessment needs to happen?

on 13 May 2008 at 6:24 pm2 Dan SchellenbergBetter yet — simply give them both triangles, and ask them to solve for the unknown side. No headings. If you have already taught them how the law of sines and cosines came to be (namely, creating an appropriate right triangle and solving it as they already know how to), I find the vast majority nail it. This is particularly true if you get them to describe the ‘laws’ in pattern form instead of using variables.

Additionally, breaking up assessment into small little chunks makes it so much less authentic. If I have a unit on, say, solving triangles, I should be able to solve any given triangle. How is knowing how to solve only a ‘law of sines triangle’ going to help me? Where’s the higher level thinking here? If I say “Hey, use the law of sines on this one”, and then assess if they can punch the right buttons on the calculator, have they truly learned anything? I would contend not. If however, I can give them any old values for a triangle, and they can figure out if such triangle(s) exists, and solve it, now they have shown some thinking. I cannot buy into an assessment system that does not allow for that type of learning.

on 13 May 2008 at 6:34 pm3 sam shahHi Dan,

I recently taught law of sines and cosines, so I know the struggle. Especially with the two solution ones with the law of sines! Anyway, I know you’ve talked about the issues of synthesis when it comes to your assessment methods, but I’m thinking about your Slide 1 and Slide 2 above:

The difference between the first slide and the second slide is that students need to make a decision as to which law of use. Why is one appropriate for a particular triangle, while another one is appropriate for a different triangle; then they have to solve.

Your disaggregated assessment, I like it. But it lacks that decision making on the student part (you tell them whether to use the law of sines or cosines). But I think you could even check that bit of their knowledge too (and maybe you did this anyway)? You could give them a triangle with some sides/angles labeled and ask them to decide which law was more appropriate to use (but not solve).

The more I’m thinking about how I’m assessing next year (definitely not the same as this year), the more I’m thinking you’re really onto something great here.

Still, the synthesis thing is hard. I wonder how I’d label the problem:

Maybe I don’t need to, maybe saying “quadratic trigonometric equation” is a small enough chunk. But some of these questions require knowledge of: the quadratic formula, the unit circle, reference angles, quadrant sign analysis (those are the bits of knowledge they need to do these problems).

Well, no one said it would be easy!

on 13 May 2008 at 6:37 pm4 sam shahHi Dan,

I recently taught law of sines and cosines, so I know the struggle. Especially with the two solution ones with the law of sines! Anyway, I know you’ve talked about the issues of synthesis when it comes to your assessment methods, but I’m thinking about your Slide 1 and Slide 2 above:

The difference between the first slide and the second slide is that students need to make a decision as to which law of use. Why is one appropriate for a particular triangle, while another one is appropriate for a different triangle; then they have to solve.

Your disaggregated assessment, I like it. But it lacks that decision making on the student part (you tell them whether to use the law of sines or cosines). But I think you could even check that bit of their knowledge too (and maybe you did this anyway)? You could give them a triangle with some sides/angles labeled and ask them to decide which law was more appropriate to use (but not solve).

The more I’m thinking about how I’m assessing next year (definitely not the same as this year), the more I’m thinking you’re really onto something great here.

Still, the synthesis thing is hard. I wonder how I’d label the problem:

Maybe I don’t need to, maybe calling it “quadratic trigonometric equation” is appropriate. But some of these questions require knowledge of: the quadratic formula, the unit circle, reference angles, quadrant sign analysis (those are the bits of knowledge they need to do these problems).

Well, no one said it would be easy!

Keep on keeping on!

on 13 May 2008 at 6:40 pm5 Mr. SadlerDan,

Usually I agree totally with what you are saying but I have to argue a bit with this.

If you are just assessing their ability to use the mathematical skills of sine law and cosine law, then splitting into separate problems that say “Please use sine law to solve this triangle” is appropriate.

What I find to be the interesting part of math though is getting a collection of tools, and then choosing the appropriate tool to solve the problem (which I suppose is testing something else). Perhaps it would be best to give three questions…one that tells sine law, one that tells cosine law and one they have to decide for themselves?

This gets into the debate is it better to teach the skills or the concepts, and I don’t want to touch that one tonight…

on 13 May 2008 at 7:24 pm6 danThe math folks here who argue I’ve specified too much and basically handed the students the right tool are pretty close to the mark.

To do this right, I’d have to hand my students several triangles, in no particular order, and let them decide which tools to apply, whether one of the sine/cosine laws or simple trigonometry.

But this is really important: I have to maintain separate rankings for each of those concepts. Because calling this sheet of paper “Triangle Trigonometry Test” and tossing in 73% or 62% or 55% leaves me no purchase later in the month when I’m remediating.

This isn’t to say there aren’t problems with my assessment strategy, simply that, in this case, I think we can work this out.

on 13 May 2008 at 7:25 pm7 danYeah, after thinking about it more, I’m definitely noting this down for next year.

I’llknow which problem maps to which concept, but my students won’t until we record them the next week.on 13 May 2008 at 8:26 pm8 TonyDan…that last comment of yours nailed it. YOU know the skill, but they don’t.

on 13 May 2008 at 8:33 pm9 Jason DyerI do start teaching this by separating out the Law of Sines and Cosines and directing them as explicitly as you do. Nothing wrong with that.

When they’re ready to start mixing, we dive directly into applied problems from there (so they aren’t even necessarily using just one or the other for an isolated problem, but possibly the law of sines followed by the law of cosines followed by a geometry trick). I figure if I don’t let them see the point of learning the thing soon after they’ve got the basic idea it feels to them like treading water.

I also tell them that the Law of Cosines is on my Top 5 List of Things I Wish I Had Remembered From High School When I Got To College Because the Professor Assumed We Were Experts.

on 13 May 2008 at 9:03 pm10 SarahI love the idea of students not knowing which problems go with which concept–reviewing this week I’ve heard way to many students say “but I don’t know what I’m supposed to do” if I don’t tell them the concept before the problem.

Having the motivation for mastery intertwined with the assessment system seemed key in your original writeup. Next year, how will students test out of a concept?

Johnny’s gotten a 5 on Law of Sines, but a 3 on Law of Cosines. Will he still be able to cross off concept 26? Will he have to be able to label the problem as the concept? Will you still give the concept number on the top of your quizzes or not?

(I’ve adopted the concept quiz strategy for some of my classes this year and need to pull it off better next year. So these questions are one’s I’ll be trying to sort out too.)

on 13 May 2008 at 9:25 pm11 danAlright, so on more problems than not, I’ll give them the concept name. If I write Similar Figures at the top of a complicated problem involving shapes that are similar in notation only, it isn’t going to matter. I mean, the tilde is there.

Still and all, this bugs me:

Presumably, if Johnny has passed Law of Sines, he’ll recognize it and not do it. Presumably.

on 14 May 2008 at 3:17 am12 Brian CormierThe way I see it, these are 3 separate skills: using the Law of Sines, using the Law of Cosines and knowing which law to use for which problem.

on 14 May 2008 at 4:03 am13 Frank N.I agree with Brian. You could make another concept/skill called “Triangle Trig” or something like that and have that be an additional question on your weekly assessments. I could see an addition question like this for a variety of skills–a kind of umbrella question which pulls everything together and tests the ability to synthesize and apply. You could still give each concept a name, which avoids the no-name issues previously mentioned. And still, both you and the kids would know that they could crank out the Law of Sines, Law of Cosines, and right triangle trig, but that they can’t figure out which rule to use when they aren’t told. Then you can hone in on THAT skill during remediation.

on 14 May 2008 at 10:07 am14 PeterI do the same thing as you do with integrals (searching primitive functions). My pupils need to know 3 techniques. For each of these they get a problem and get the technique they need to use. Then there are 2 problems where they do not get the technique.

Furthermore I use your how math must assess idea. So i have 4 goals to test. One for each of the techniques and one for the mixed form (this is worth double the points).

on 19 May 2008 at 6:01 pm15 Some Places To Visit On The Interweb « Continuous Everywhere but Differentiable Nowhere[…] On Nailing/Blowing Assessment (dy/dan) […]

on 19 May 2008 at 7:07 pm16 IB a Math TeacherOne way to assess what the students know is to just ask for a particular triangle, “Is it more appropriate to use the Law of Sines” or “Law of Cosines” to solve for x on the following triangle?”

With it being between only two choices, you might have to have a few on there so you know that they understand what to use and when, but at least that would give you an indication that they know which to use.

Then you can add problems to see if they know *how* to use the Laws.

on 09 Jun 2008 at 8:43 am17 TheInfamousJ:: mulling over this one ::

My personal teaching philosophy is that the skills are a toolkit and, due in no small part to my own nightmare that was high school where I didn’t realize I had to carry the skills forward and not just dump them, I try to teach my students in a cumulative manner. They have to decide on the mechanism for solving the problem using each of the skill pieces that we’ve been reviewing.

If I were a math teacher, I’d teach law of sines and law of cosines, but then would spend a lot of time in class on a triangle with two sides labeled and how to create the mechanism for solving the problem where the first step would be identifying which piece of the toolkit (law of sines or cosines?) should be used and the second piece would then be successful execution. I call it a “game plan”.

I think what I am going to try next year is concept quizzes as quizzes, with grade replacement. And then on my test it will be the (Bloom’s taxonomy) evaluation piece of creating the correct game plan along with the proper execution of the individual pieces which the students should have taken ownership for learning, as there will be at least two if not more quizzes before a test.

I’m not yet ready to give up tests. It is not that I like aggregation so much as the fact that the skills of chemistry require aggregation. In science, the ability to break down a larger problem into smaller, easily handled pieces (identifying variables to turn in to an experiment from a complex observation, for example) is at its very heart.

On the other hand, I’ve seen an Alg 2 teacher’s grade book (his desk is beside mine in the faculty workroom) and it does read in terms of chapters. So I completely see where you are coming from, Dan.

on 29 May 2009 at 7:41 pm18 SarahDavid has me digging though old posts and reflecting on the past year.

I kept concept quizzes this year, and kept the clear delineation between skills. Did you end up switching to “I’ll tell you what’s on this sheet but you need to figure out what you need to use on each problem”? Any updates on how that went? Either here or over at the current discussion. Thanks.

on 29 Jun 2010 at 7:46 am19 My SBG Journey « Action-Reaction[…] I was bummed when Regents exams scores stayed flat under my Dan-Meyer-inspired system, but the increase in student morale made it worth keeping. So I used it the following […]