## Making An Honest Woman out of Assessment

These are satisfying times to assess the way I do. We’ve got Dead Week starting Tuesday, finals the week after that, and then grades are locked. Students are panicked, but the way I assess, they can focus that panic.

“How do I get my grade up?” they ask.

Most of the time they bring their Concept Checklist [pdf | cwk] along. If it’s filled completely, I know they’ve bought into the system.

Otherwise I pull up their assessment scores in PowerSchool. Several completed concepts. Good.

3 out of 4 on Triangle Congruencies. Less good.

“Have a seat,” I say. “Let’s talk about it.”

Sometimes I’ll tutor them. Other times they come prepared, studied up, good to go. Either way, I’ll scribble down a one-question test on Triangle Congruencies.Â  A decidedly un-scribbled example:

They pull a higher score. I immediately replace their grade and report the new percentage. Good times had by all.

The resource teachers are out on the prowl these days, too, trying to push their case kids through challenging courses. “What can Robbie do to pass?” they ask me.

I hand them a sample exam of every concept we’ve taken this semester. “If Robbie shows me he can pass Solving Proportions and Evaluating Expressions, I’ll forget the classwork and the homework he’s missed.”

I don’t make a huge deal of it, but I don’t care about the final either. By the end of Dead Week, I can tell you within a tiny margin how well every student knows every concept. Such is the power of this assessment strategy, that it renders even the largest test of the term inconsequential.

Attached:

1. Concept Checklist [pdf | cwk]
I'm Dan and this is my blog. I'm a former high school math teacher and current head of teaching at Desmos. He / him. More here.

1. #### Todd

January 13, 2007 - 8:12 am -

“I’ll forget the classwork and the homework he’s missed.”

Then how important was any of that to begin with? If it can all be replaced with a single assessment, then why assign it in the first place? Why don’t you just let all of your students wait until the end to pass a single assessment instead of allowing them to bust their butt all semester long, yet earn the exact same grade as the slacker who knows how to do it, but chose to wait until the last minute?

Remember that teacher who does a terrible job and earns the same amount as you do? Remember how pissed off we both are that we work in a system that allows for that? I’d imagine a crop of your students feeling the same way about that kid who got to take a single assessment on each standard and pass the course.

I’m not a fan of finals. I think high school doesn’t need them and they are a relic clung to by the typically sedentary educational community. But I am a fan of sustained effort through the course of a school year. Grades should reflect that.

2. #### Todd

January 13, 2007 - 8:14 am -

P.S. Nice to see the attachments! I’ll bet that you’ve made life much easier for at least a teacher or two somewhere in the future.

3. #### Rich

January 13, 2007 - 8:52 am -

Dan,

Thanks for a) being transparent with your philosophy of assessment and b) giving us the details of how you implement it. I have a couple of questions that might tend to fit into the “detailed” end of things but hopefully you’ll indulge me.

First, you have frequently mentioned a four-point scale, for each skill category on each assessment. I’m curious whether that’s 4 points for one question, 1 point each for 4 questions, or more likely something in between depending on the specific topic. For example, the triangle congruency problem you included with today’s posting has four parts — enough to give you, the teacher, a decent indication whether the student understands SSS, SAS, etc. sufficiently. Would “problem 9” (your example) be worth those 4 points, toward that concept in the student’s concept checklist?

Second, how do you handle the student who struggles for a fairly long time with a particular concept – while your rolling selection of concepts (in the weekly quiz/test) has moved on? That is, if a student is still stuck on concept “C” but your weekly test is now covering concepts F, G, H, I, and J? You mentioned coming up with a one-question test on the spot – would you handle this situation this way? A variation of my question would be for a student who has missed one or two (or more) weeks’ worth of tests – would you pull out the old ones for her to use one she returns, or come up with new/unique problems?

Hopefully both of my questions make sense – I am extremely interested in converting my own (middle school math) assessment and grading approach next school year. Right now I’m “stuck” in a fairly traditional model with a homework weighing pretty heavily, but I can spend the rest of the current school year and this summer aiming toward a change for ’07-’08.

Again, many thanks for your clarity and transparency, and for a darned well written blog at that – Rich.

4. #### dan

January 13, 2007 - 9:30 am -

So many objections, so little time. You’ve got some word choices in your first paragraph that indicate a point of view I may not be able to reconcile.

Rather than take your points one by one, let’s find the destination of your POV.

Let’s say I can open the Geometry textbook to any page, I can point to any standard, and this kid can demonstrate total understanding. I do this, in fact, by breaking every course down to forty standards-based concepts and asking my students to prove their knowledge twice (not “a single assessment”), on problems of increasing difficulty.

Let’s say this kid didn’t complete any homework and sat back on classwork.

I pass him w/o hesitation. A grade no lower than B-.

You fail him. You make him retake Geometry, a class which he already knows start to finish.

What’s he doing in there the second time around? Developing study habits, I suppose? Learning the value of hard work? Becoming a better citizen?

And this is where we split, I reckon, but unlike other disagreements we’ve had, I’m willing to put a moral negative on your stance here. I think holding the kid back does him a grave disservice.

Nowhere in the content standards for Geometry does it list: “Students complete work the teachers assigns.” Lots of stuff in there about Geometry knowledge but nothing about the day-to-day grind of being a student. If a student can demonstrate competency in every standard, I don’t even see how it’s lawful to hold a student back.

So you’ve now got philosophical and legal basis for flipping homework the bird. How about anecdotal?

I have five students in my third period Geometry, all failed by the same real good teacher who has too much affection for the virtues of homework. All these kids are acing my course now.

The concept checklist above represents one of them. He resents having been held back, which forces some ugly class management issues. Over this entire semester, I’ve taught him nothing.

Why would you hold him back?

5. #### dan

January 13, 2007 - 1:44 pm -

And now to Rich:

I don’t talk much about the logistics of this system in my papers / presentation, mostly for worry of getting too deep too fast. Assessment reform remains my #1 priority. Anything that’s unclear, though, please bring it up here.

At this point, first semester in Geometry, I’ve got fourteen concepts running. They’re all listed in the concept checklist there.

First, they’re all worth four points. The more committed to this model I became the more I went out of my way to make the 4-point scale work. This meant a lot of two- and four-part problems. Each fourth of Triangle Congruencies there was worth 1 point.

Even though you didn’t ask, it’s a good time to mention this: what do you do when a student gets a perfect 4/4 on the first set, and then a 4/4 on the second test (achieving “mastery”)? You need some way to tell this student apart from those who only scored that first 4.

So in whatever grade program you use, you increase the total points to 5 on the second assessment. 4 + 4 = 5. This is great now because that student who pulled a 3/4 on the first (easy) problem used to have a 75% but now has a 60% with 3/5. This means that unless you want to ride a D- on Triangle Congruencies the whole semester, you want to come in before school or at lunch to get some 4’s.

This system is built to address your second concern. A student never really understood Concept #3. She comes in before school / at lunch. You tutor her. You scribble down a question for her. She completes it. You update her grade. Everyone’s happy.

For the student who’s missed lots of tests, I used to take the time to construct new tests, but nowadays I just give the old ones. My students just aren’t devious enough to game the system like that.

Hope this clears a few things up.

6. #### Elona

January 13, 2007 - 4:57 pm -

I’m intrigued by your form of evaluation. I have a question. Do your students still write that dreaded final exam that is worth 30% at my school. Many of my students do not do well on the final.

7. #### Todd

January 13, 2007 - 5:40 pm -

He’d fail the course because our report cards are not standards based. Our school has a homework policy; there’s more to a course than standards. I suspect that’s the core of our ongoing philosophical disagreement.

8. #### dan

January 13, 2007 - 7:46 pm -

Elona: I’m not sure I get your post. If you’re asking if my students take the final seriously, then, yeah, they do. 10% isn’t anything they’re willing to blow off.

Todd: Seriously? You’ll dismiss all of that simply because standards crop up? Lose it then, if it’s too loaded, because this isn’t that debate. Replace “standards” with “knowledge.” A kid comes to me. He knows the content of a class. He didn’t have to work for it. What profit is there in holding him back? To make things “fair” for the kids who needed homework to understand?

9. #### Todd

January 13, 2007 - 8:20 pm -

No, not because standards cropped up. Like I said, there’s more to a course than standards. It’s because I believe in teaching a course designed to move a student forward, to increase the amount of knowledge he or she had when entering. Because of the nature of the course, that’s entirely possible and responsible in an English class. It may not be in a math class. “He knows the content of a class.” That doesn’t really apply to English.

Math and English are different in this respect. English has a set of standards for every 2 years, which of course means there are only 2 sets of standards for use at the high school level. There are no clearly defined exit criteria for each year and the standards are nowhere near as specific and demonstrable as math. I can’t apply the standards the same way you do; English standards don’t lend themselves well to finite testing periods or anything remotely resembling a checklist (not a dig, I mean that sincerely). This is the reason I love, and hate, teaching English.

10. #### Shelli

January 14, 2007 - 2:18 pm -

How do you handle the students that “rent” rather than “own” the material? I have a lot of students that can pass a test, but 2 weeks later have little retention. How do you promote the retention/spiral learning (other than through a final).

I really like the ideas presented here, but I’m trying to figure out the logistics/classroom management side of it. Do you have one day a week that you’ve declared quiz day where kids can demonstrate their compentency?

Thanks!

11. #### Elona

January 14, 2007 - 2:52 pm -

Shelli, I find displaying the kid’s work on the walls of the classroom helps kids remember more of what they have learned. When the kids tune me out and are busy looking around they “accidentally” review the old concepts on the posters, charts graphic organizers, mobiles etc (I’m a great believer in projects) that they have created. Sometimes when I have to share a room, I only have one wall to do this but if it’s my own room then I have all the walls covered. Even high school kids love to see their work up. Sometimes I let them put their own work up and sometimes I put it up. When I do take the work down, the kids really notice. I put up all work, not just the work that got the A. Kids look around and see how to improve their work. How do I know this.? Kids have told me this. I also post giant mind maps that review the unit as we do them. Can’t ya tell I’m a visual, kinesthetic learner. I haven’t figures out what to do for the auditory learner. Maybe we need to write songs and play them as the class comes in and gets settled- I’m only half kidding.

12. #### Shelli

January 14, 2007 - 3:31 pm -

Elona,
I would love to do that, but I can’t. Besides sharing a room, they also use my hallway for all testing (SAT, ACT, etc) and we cannot have anything on the walls that involves the curriculum. I am also a visual learner and use a lot of projects, so I fully understand where you are coming from.

As an example of what I meant in my earlier comment, I teach Geometry, which at my school is between Algebra 1 and Algebra 2. There is an End of Course exam for Algebra 1 that the majority of my students supposedly passed. However, they have Algebra Reviews that are due every Friday to help keep their skills current for Algebra 2, yet many of the students can do 20% of the work on each Review. (And that is with posted hints on the back board for the Review!)
*sigh*
Shelli

13. #### dan

January 14, 2007 - 7:00 pm -

Whoa … Shelli’s got a math teacher forum running. I’m definitely going to spend more time there than I should tonight.

Regarding your question, you’ve probably struck the sorest nerve for me about this whole assessment set-up. I spiral the concepts for six weeks at a time (a twenty minute test every Friday) but it’s still an issue.

Because once students have mastered a concept (passed an easy and a hard problem) they put it out of their minds. This is great because I want them focusing on their weaker concepts. This is lousy because sometimes they’ll miss an easy opener problem that, by the system, they’ve mastered.

I’ve had to tell myself that “mastery” isn’t equivalent to “perfect recall forever.” “Mastery,” as I’ve defined it anyway, means I or a textbook or a student buddy can give a few cues to the offending student, and the whole concept comes flooding back.

From my experience, the alternative — where students don’t have a Concept Checklist, don’t have a roadmap to navigate the sheer breadth of math — yields far worse retention. Improving retention will always be on the to-do list, however.

14. #### Shelli

January 14, 2007 - 8:10 pm -

Dan,
I hope you enjoy the site :) It’s been a several year process and I’ve truly enjoyed working on it.

I’m wondering if you can merge the two assessment ideas together and keep everyone happy.

1) The content mastery side of it, where the students keep their mastery list and do the component quizzes. The “grade” goes in at the end of each marking period as to how many components they have mastered so far.
and
2) The more traditional assessment that is cumulative at all times. This way you would have some spiraling and the focus on retention because chapter 5’s test will also cover the previous curriculum. This would also help the issues that some of us would face with gradebook programs that are mandated by the district and not flexible on grading.

The homework issue will never be resolved IMO – I don’t put much stock in homework grades, I prefer their grades to come from work that I can monitor in class.

15. #### dan

January 14, 2007 - 9:58 pm -

Re: Homework, agreed.

Re: Assessment, compromise is a good, teacher-ly suggestion, and has worked in several classes I’ve observed.

There was going to be a “but” here, but, nah, nevermind. It works. The compromise works well.

16. #### Shelli

January 15, 2007 - 5:57 am -

I’m not sure if I should be offended at the “good, teacher-ly suggestion” part… (just kidding, I don’t take offense easily).

I would like to hear the “but” part though, your mini-thesis was very interesting and I would like to hear/discuss more of your ideas.

17. #### dan

January 15, 2007 - 2:38 pm -

I just worry compulsively that a student’s grade accurately reflect her understanding, particularly along the sharp line of passing/failing. These large cumulative tests are functional but I think of the student who fails test after test at the start of the year only to find his motive halfway through, or the student who was riding a C all semester long only to develop an A-level understanding with the help of a review unit.

What do they do with their low cumulative test scores?

Unless you’re constantly issuing retakes of these large cumulative tests, they’re stuck with their low grades even though their current conceptual understanding merits a higher one.

Teachers that lean on cumulative assessment also have a hard time using their tests for differentiation. (What does a D on “Chapter 6 Test” tell me about your knowledge of an individual Chapter 6 topic?) By combining the two systems, you’d take care of the differentiation issue, which is great. But there is that other concern.

18. #### Shelli

January 15, 2007 - 5:02 pm -

I understand what you are saying and I agree that it is a concern, we need to decide if grades reflect their ability to follow rules and learn on our timetable or if we want their grades to reflect their ability on the day we issue the grade.

I also agree with the differentiation issue (which is a big buzzword in my district right now), but one of my concerns would be (based on the quiz you posted), it seems that the assessment is one concept at a time. Many students (in general) can “master” one concept, the problem occurs when students are presented with several concepts at a time, many of them do not know which tool/concept to use at the appropriate time and the application of those concepts. That’s why I would want the cumulative element to be there, or ideally application-based problems that combine several concepts into one problem.

(Granted, you may have this issue worked out already, I’m just commenting based on the 1 quiz posted)

19. #### dan

January 16, 2007 - 6:12 pm -

I usually test six concepts at a time. Sometimes three, if I want some free time that weekend. Even then, that same concern weighs on me. Always. If I didn’t increase the difficulty each subsequent test, I know the system would be too easy on the students.

20. #### Lsquared

January 17, 2007 - 12:40 pm -

I do a similar thing with testing derivatives–students have several weeks during which they can retest to increase their score. The thing I don’t do is tutor and then immediately test. I’ll test, and then tutor based on their performance on the test. Then I’ll retest the next day. I know that’s not enough to ensure that they can do it in 2 weeks, but it’s my attempt to make sure it gets at least partly into their long term memory.

21. #### dan

January 18, 2007 - 4:51 pm -

I feel convicted. Each day during this “dead” review week I’ve selected some low concepts, tutored, and then re-tested them. I need to wait a day. I need to wait a day. Crud. You’re right.

22. #### Robert the "Nerd"

January 18, 2007 - 10:21 pm -

I think 4+4=5 makes perfect sense. I have followed this format in my math class for several years with a variety of students at various levels of performance and they remember better, are more focused, and more conscious of thier responsibilties for learning than thier counterparts who use a more traditional form of assessment. Also, if you are a bit unscrupulous (SP?) you can just tell the kids that each time they take a concept test they must take each question whether they have a five or not, forcing them to stay current. How can you do this you say…well kids need things from teachers that are not grade related and you can threaten/tempt them with these enticements until you are certain they have placed these topics in long term memory…miss ya DAN!!!