I think the main value in those types of complicated problems is the assurance that the student understands things when she or he answers them correctly. The risk is not knowing which part they missed. Perhaps you can assess first on isolated concepts, and then have an “integration” concept that involves piecing together other concepts?

Not sure that I’m adding some brilliant insight to your posting (or the two previous commenters), but I definitely do recognize the care that must go into creating a good problem for assessment, and how we can unwittingly create complicated problems that prevent us, the teachers, from truly seeing where there are needs and where there are strengths. For example, my sixth graders just finished multiplying fractions — but even before we quizzed on that, I was realizing that their greater challenge lay not in the multiplying of fractions, but in simplifying the results.

If our assessments really ARE designed to help inform us and our students about where they are strong and where they need some more work, then we must create appropriate problems to “diagnose” that (to use mrc’s word). But I do also agree that we can’t dismiss multi-layered problems altogether, since rarely in the *real world* do we come across naked, isolated mathematical problems. Steve makes an interesting point about assessing first on the specific concepts, and later assessing on integrated concepts.

Just wanna say I enjoyed reading your “How Math Must Assess” manifesto, even tho I teach EFL which is much, MUCH, harder to break down into unit skills. Also, there’s the added variable of performance because learners may know something but be inhibited by performing it in front of their peers (yet if they don’t perform it, how do you know they know it?), and there’s also the testing of passive versus active knowledge, all elements which are probably foreign to math teaching. Still, I think your post did its job, i.e. getting me to think again about assessment.

PS You might get something out of Ed Nuhfer’s articles. Here’s a sample. He’s prompted me to re-think my teaching philosophy, not just in the area of assessment. He’s university-based, but a science teacher, so you may glean something.

You say: “Did she fail because she doesn’t understand the Pythagorean Theorem or because she doesn’t understand Cones?”
I am curious as to what exactly she is supposed to understand about cones. Your problem asked her to find surface area of the given cone. To solve it, she needs to know the formula for surface area. To be able to evaluate it, she needs to know what in that formula corresponds to quantities given in the figure. Which part of this problem then relates to her understanding of the cone? In other words, I would like to know what is being assessed by this problem.

I was actually thinking about what it is that we deem important that our students know, not about changing your assessment. Personally, I don’t find value in their knowing the formula for surface area of a cone by heart. In other words, this is not something I would want to assess (I don’t know however whether the standards require it). For example, did they see the derivation of the formula? If so, was it insightful? Did it bring anything new to their understanding of cones? Or to their understanding of area? Or to their understanding of …? Contrary to the cone example, I can see how and why we would want them to know surface area of say a cube. Not by heart mind you, but by knowing what surface area is, they’d be able to find the formula and hopefully evaluate it. I am not sure that we can say the same for the cone (at least not without significant amount of work). So, to humor me, let’s take the knowing (memorization) out of this assessment. That leaves plugging in the quantities and interpreting the answer. I don’t think either one of these tasks is trivial. Or should I say, the demand of each of the task depends on where these students are in their mathematical education. Evaluating functions and making sense of the answer you got is, I believe, an important part of what the students should learn. However, correct evaluation of a function is certainly less impressive if a student in a calculus class does it, than if a middle school student does it. That type of consideration determines whether it’s a B- or an A or a C answer, and this is something you’d know better than I do.

I would also like to agree with Rich that although “simple” problems easily indicate where exactly students may have troubles, rarely do the real problems come isolated in a single concept format. Besides, one of the things we need to make sure is that our students can make connections between various ideas. I know that we can say that we do it in class, but the problem in that is that we can’t truly know how each of the students made those connections.

This was the only thing that bothered you? Good. First, I thought you hardly ever gave homework, and I don’t know whether you look through the ones you do give. But I would venture that homework would be the next best way to determine how well they make connections, by themselves. As for the other forms: openers / classwork / conversations, I personally, find it hard to assess ALL of my students that way. Why? Some students don’t speak as much as others, some are shy. We may think that all of our students participate equally in classwork, because we call on all of them, but that usually isn’t the case either (try to pay attention, or have somebody else visit and write down how many times each student has spoken during 1, 2 or however many classes). Let’s say that you make a point to ask a question of each student (can you call on each in one period?). Some will be able to give you answer immediately, some won’t. If they don’t know the answer immediately, how much time do you have to figure out what exactly gives them problems? And even when you’ve gone over something with the whole class, does it mean they all got it?

I’m just saying it’s not all black and white. Not all your problems should be simple, and not all your problems should be complex. Balance. I’m for balance.

Mid to high 30s are huge classes in my mind. Mine were in mid 20s and I still had to struggle to make some people talk. Maybe I suck. I wasn’t saying you should remove the problems you have now, but you may choose to add the other ones in. If I am not mistaken that wouldn’t damage your ability to “target area of weakness”, but would allow for assessment of other things as well.

Dan, your blog has helped my teaching more in the last 3 years than all professional-development done at my schools combined, however, I disagree with the procedural nature of assessment items like these. To be honest, the cone example you gave seems more like an “exercise” than a “problem”.

It seems like you value–above all else–the ability to look at an assessment score and know exactly what that number means. I like that and would like to find a way to do that. In doing so, however, it seems that you break everything down into such compartmentalized, small individual skills, that students are never assessed on their ability to solve rich, layered, challenging mathematical problems that require critical-thinking skills and/or require them to communicate their thinking.

Perhaps I’m assuming too much. Perhaps you have changed in the 2 years since you initially posted this entry. Are you assessment items still this procedural? Also, do you give Problems of the Week or portfolios or projects or any alternate ways of measuring what kids know?

[Thanks for your blogging. As I said earlier, reading it (and the comments) really improves my teaching abilities.]

In structuring assessment, I try to balance procedural exercises with richer problems that attempt to assess how well students are able to decide what tools to use. I believe both types of skills are important in determining success at the next level.

This year in my PreCalculus class, I have many students who are excellent at solving exercises such as “find the roots of the following quadratic trinomial,” where students are asked to apply various skills in isolation. (My quizzes, which are worth 25% of the course grade are made up of mostly these skill-based questions) However, those same students genuinely struggle with word/story problems where students must decide what information is relevant and decide which skills to apply. (Much like the excellent situations and problems you give us to consider in your blog) While I try as much as possible to include these in classwork, I also try to show that these problems and the problem-solving and questioning skills developed by working on these problems are important and worth measuring by including them on more formal assessments.

I try to make it so that a student earning an “A” in my class has demonstrated mastery of basic skills and has demonstrated the ability to know in what context those skills should be applied. Many times students ask why bother taking advanced math, when they might never factor a polynomial or solve a quadratic equation ever outside of math class. For many students, the critical-thinking skills and communication skills they build by figuring out how to apply the procedures they’ve learned is the main skill that they’ll draw on when it comes time to communicate detailed, techincal information in the future.