On Getting The Concept Checklist Wrong These Last Six Years

May as well get this out of the way as long as I’m in this public state of contrition.

The concept checklist, in theory, is where students track their progress towards mastery. They write down concept names in rows as we test them and then record their scores (on a four-point scale) along that row, one after the other, each time they retake a concept quiz. I log only their highest scores in the gradebook and whenever they record two perfect scores on a concept, they never have to take that concept again.

The concept checklist is a mess. I run through the same script every year, illustrating the same process with better and more precise visuals every year to no avail. The process confuses students. The process puts students farther from meaningful self-assessment not closer. I saw another checklist crumpled in the trash last week and figured it out.

Their highest score matters much more to me than the specific ordering of low scores preceding it. So forget the earlier low scores. Students add length to the bar as they improve on earlier scores. This checklist design is consistent with our class ethic that “what you know now matters to us more than what you used to know,” whereas the other design maintains a permanent record of “what you used to know.”

So here’s an updated attachment.

BTW: Reader Jacob Morrill does me one (or two or three) better with his adaptation, which is superbly designed:

About 
I'm Dan and this is my blog. I'm a former high school math teacher and current head of teaching at Desmos. More here.

51 Comments

  1. I’m stealing this, just like I stole the original.

    But I’m going to split the rows on the score tabs, so they can keep track of two top scores.

  2. I like this. It also adds some score-keeping meaning to the stamp, whereas before it was just a [meaningless?] piece of praise to validate what was already written down (I wonder what Alfie Kohn’s thoughts on that would be). Here, if there’s no stamp, there’s still one more to go.

  3. I’m sharing this with the folks at my school as we discuss keeping track of standards or goals. This seems like a nice way to implement for students.

  4. This is great! I actually use something similar to this with my 4th graders, but you’ve gone up and improved it.

    Going to borrow. Thanks!

  5. I really like the elegance of a “personal best” record that obscures the student’s score. Adding another block to the “meter” means you’ve beat (or tied?) your best work, and it is always good news.

    Is there a level on the score-free chart that denotes mastery? Is three improved scores (reasonable on a 1-4 scoring system), or two perfect scores, or some some other metric entirely. Does reaching mastery have any affect on what they’re asked to do in class?

    As someone who’s mental proceses were warped by video games from a young age, I often try to build class routines that mimic game logic. I’ve used various forms of super meters and XP bars, and this is a nice addition to that toolkit.

    Thanks!

  6. Ok, that’s funny. I used this for my practicum last fall and assumed it was supposed to be a fill-in-the-boxes template to begin with.

    (Of course, being a practicum, the students were mostly confused by what I was doing anyway as I was changing things up two weeks into the class.)

  7. Do you no longer require two separate “perfects” in your quiz system?

    Would you track this by adding a 5th column and labeling it “4+” or “5” or “Heck yea you did it it again!”?

  8. Jacob, I would guess the stamp is supposed to be the “perfection” mark (note how the third row doesn’t have a stamp yet, so the student only scored a 4 once).

  9. Incidentally, I did try a microassessment system in my AIMS Math class (seniors who haven’t passed our standardized test yet) but the student feedback was overwhelmingly negative. I’m still not sure why, but I think it is partly that they were seniors and set in their ways. In any case I have shifted to a more standard quiz system.

    The other problem with these students in particular is that “what they know now” is a moving target; it’s very hard to presume mastery even after multiple demonstrations.

  10. Do you still track past scores in your own records? I waffled on that question when I started using this. “More data -> more analysis” is good; but “less data -> less administrative hassle and tidier mark book” is also good. Plus if I really don’t care about past scores then tracking past scores is kind of … weird? Hypocritical? Except not really so whatever.

    (I tried setting up a spreadsheet to get the best of both worlds; got something close enough to use but still felt like I needed to hunt down a few more spreadsheet tricks to make it really click. Or just build a database to do it, maybe.)

  11. If I’m following you, shouldn’t your sample student only have one box filled in for concept 4, since the second and third scores are lower than the first score?

  12. I don’t get this at all. Why don’t they just write down the dates when they get a 4. If we don’t care about past lower scores, why write them down?

  13. Gotta get Jason Dyer a dy/dan merit badge for fielding questions so well. Best I can do on short notice is to urge everyone reading to check out his most recent post outlining a killer intro to area. (Kicking myself for not coming up with the same intro to units.)

    I figured, when I posted this, there’d be some folks like josh g. who saw the original concept checklist, misinterpreted my instructions for it, and used it the right way since day one. Thanks, bud, for letting me wander so far off the trail these last six years.

    Andrew: Is there a level on the score-free chart that denotes mastery? Is three improved scores (reasonable on a 1-4 scoring system), or two perfect scores, or some some other metric entirely. Does reaching mastery have any affect on what they’re asked to do in class?

    Mastery means they get a perfect score twice. Two 4’s. After the first four, there’s nothing more to shade, so on the second four they get a stamp.

    Carol: If I’m following you, shouldn’t your sample student only have one box filled in for concept 4, since the second and third scores are lower than the first score?

    I give them credit for their highest score.

    Perhaps I’ll take this moment to direct other questions to the FAQ.

  14. Seeing the different ways people interpret this is fascinating. I had students write the date in the box under their score. I don’t know why I didn’t switch to the shading in.

    I like Mr. K’s idea best though. Split the rows with a dotted line, easier to visualize the top two scores that are in my gradebook.

  15. *heh* I’d interpreted it to mean the shading/date thing from the very beginning. In fact, I’m trying to torture this into one of my Inquiries for BTSA. I *love* the idea of this assessment, and think it’s really something my kids will adore.

    (On a related note… do you ever allow local-ish teachers to observe you for a class or two? And/or do you have any conferences coming up that you’re presenting at? I’m teaching in Northern Monterey County, and would LOVE a chance to see you in action!)

  16. Interestingly, I interpreted this as what you meant all along! My skills checklists have a blank space and I encourage kids to fill them as they will – some fill theirs like a fish tank, others use horizontal bars like you’ve demonstrated above, others use pie-like slices… and some mix and match.

    I’m still working on getting a sweet “master” stamp, though…

  17. A pox on josh g. and now Elaine and Riley. I have to wonder what else of my work y’all have decided to accidentally improve.

    Elaine: On a related note… do you ever allow local-ish teachers to observe you for a class or two? And/or do you have any conferences coming up that you’re presenting at?

    I’ll be speaking at TEDxNYED, which is about as far from local as it comes for us west coasters I’m afraid. I’ll also be doing an online thing on Wednesday, February 17, right in the middle of the school day. I’ll find some way to get both of those online.

    You’re welcome to come up and hang out. I teach mornings on MWF. Just let me know and I’ll hook you up with the requisite badge, security clearance, biometric scan cards, etc.

  18. I love this, including all the different improvements being offered. It will be fascinating to see how things play out over time.

    One idea I had that I wanted to propose is — in the spirit of the Interesting Thing Followed By Big Question lesson intros — why not add a “big question”-type of question to the top of the checklist? I can’t put my finger on it yet, but something like, Which skill areas have I mastered?

    Maybe this is just dumb, but I think that a reminder question that is constantly framing the assessment question for the student can help to reinforce the process of self-assessment and lend meaning to the stamp.

    Barf away, if I’m just babbling.

  19. I’m intrigued by this and am going to try adapting it to college-level human anatomy labs. One thing I am thinking of doing is once a student has earned a master stamp I will let that student test others and pass on their own stamp of approval. I think testing others will help reinforce what the masters have learned.

  20. I use a bar graph system. They graph their scores along the x-axis on a 0-4 scale.

    http://www.scribd.com/doc/25519003/Tracking-Forces

    The big black square is the graph, it didn’t really come out well on scribd.

    On the back they just traffic light the specific standards (check/plus/minus).

    I think “what you used to know” is valuable especially for the students who are behind and only compare themselves to their peers. I spend a lot of time pointing out the first few scores to the students to point out how far they’ve come. Also I think it helps to have to point out that everyone started low, the “smart kids” didn’t start out with 4s.

    I’m curious about your reasoning for overwriting that record.

  21. JYB, I think the line ‘The process confuses students. The process puts students farther from meaningful self-assessment not closer. I saw another checklist crumpled in the trash last week and figured it out.’ makes it fairly clear the more extensive record was too confusing to the students to be useful.

    This sort of thing I’ve found is dependent on incoming student background. I’ve had classes that could handle an even more complicated version of Dan’s original form, and I have had one class for which even the above simplified version would be too confusing for them.

  22. Jason: … the more extensive record was too confusing to the students to be useful.

    This is exactly right. While I prefer more data to less, that extra data comes at an organizational cost. JYB has something pretty cool running there with what appears to be two pages per concept and a separate rubric for each.

    I can count on my students to hold onto exactly one piece of paper — their opener — and that only for two weeks at a time. I have to watch my overhead.

  23. This is so interesting. I too didn’t fully read the original instructions, and figured if I am only keeping track of the highest score, then so should the students. They were erasing and re-writing checkmarks, until one of them came up with the bar graph system! I will make sure to tell him that you approve of the idea.

    Also, as long as I am giving feedback on the system, my students LOVED it. It is an 8th grade Algebra class (I also teach Geometry), and these kids were so excited.

    I am attempting to adapt the system for a Biology class, and as you said originally, the trick is in defining what is a concept.

  24. Yes, can we have another post on how to decide concepts and design assessments? I’ve read the comprehensive resource a hundred times but I still need more.

  25. I get what you’re saying about confusing them. The topic scales I use are for everything they do, including projects, so we get pretty used to them. In the beginning of the year though it’s just crazy.

    Your checklist just screams sparklines to me. A little mini line graph running to the side.

    Or you could drop a couple extra grid lines across where the score would go for series of bar graphs. Then they could just shade them in.

    I’ve found the added value of seeing visual growth is worth the extra hassle.

  26. Yeah I’ve been reading his stuff but I still don’t know how to get away from assessing everything. Why do we teach it if we don’t assess it?

    Of course, at this stage, I’m wondering why I teach anything I teach.

    How do I get students to buy in if there is no grade attached? They don’t know the value of learning for the sake of learning.

  27. Feel I owe you some royalties. Been using the comprehensive assessment all year, the kids like it and it’s helped me a ton. A friend/fellow teacher in my district is also using it, and she shared it at the district level. The bigwigs liked it, and I’ve received complements as a result. So, my friend, complements forwarded to you.

  28. I have never used a concept checklist before, and am interested in how you format & grade assessments so that students know how to fill in the checklist.

    * Do you label each question/task with the concept it is covering?
    * Do you grade all questions/tasks on a 4-point scale?
    * Does every assessment cover only one concept?

    I guess I just don’t see how you implement this so that you don’t need to help every student fill in their checklist.

  29. Thank you for posting this idea. I revamped my entire assessment strategy with my students. They love it and I love it. I have students motivated to take tests. In fact I have had several students that normally would not have come in for extra help come in. It has also completely organized my curriculum. There is no more guessing on what you need to study to do well in my class. About the only change I made was making the rubric a five point rubric instead of four to match my style of grading. Again thanks for the ideas. This site is a gold mine.

  30. Roberto Marquez

    February 22, 2011 - 1:26 pm -

    Just started using this in the second semester. After first quiz, I like that I can break down how well class does (or individual is doing) at each concept

  31. This is an interesting thread that matches an issue that my school has just been discussing. I’m not sure if anyone is following the thread anymore since it’s been several months since the last comment, but I’ll post a comment anyway.

    At the start of a unit I provide to students the rubrics for whatever learning goals are in the unit. The rubric for a learning goal easily fits on the front of a sheet of paper and on the back of that sheet is a form for graphing scores. Actually, there’s more on the form than this. There is a spot to record the first score and a spot to record what the student’s goal is by whatever date. There is a large area to write out an action plan of what the student will do to increase performance (e.g., complete more practice, study with a friend, whatever). Below this is an area to graph the scores: the x-axis is the # of the assessment and the y-axis is the score from 0 to 4. The actual graphing could be a bar graph or line graph (I use a line graph because it better fits my Algebra 1 topic of scatterplots and regression).

    Now, this seems to be the opposite of what folks are discussing here, but the reason I like it is because the student can graphically see a trend, whether there’s steady improvement, or stagnant scores, or even if a score dips down. Following each assessment the student plots the next score on this graph and adjusts the action plan accordingly (or maybe finally commits to their first action plan). To me a process such as this is what makes an assessment truly formative for a student–he uses the feedback from an assessment to make and change what he’s doing to understand the material.

    Nothing about a system that maintains a record of only the highest score achieved precludes the response to feedback procedure that I’ve described, but the visual depiction of a trend would seem to have more power to influence a student’s habits.

    But, perhaps I’m totally wrong on this and your system is superior. Certainly, your system has a compactness to it of putting all learning goals on one sheet of paper. In my system each goal’s rubric/graph is on its own sheet of paper. My gradebook software does print a report of all of the learning goals; I’m only talking here of the manual monitoring of learning that the students themselves are doing on paper.

  32. Kevin: Nothing about a system that maintains a record of only the highest score achieved precludes the response to feedback procedure that I’ve described, but the visual depiction of a trend would seem to have more power to influence a student’s habits.

    Love the graphical approach. Obviously the downside is more paper, but the upside is considerable. Thanks for sharing.

  33. Anyone using PowerSchool with this? Any advise? If I give 3 points out of 5 that’s a 60%. Doesn’t sound like what we’re going for. Also, is partial credit out the window? Finally, I can’t help but wonder if maintaining records and logging a bunch a scores for each test/assessment doesn’t take an insane amount of time. May sound selfish, but I’m attempting to be realistic.

  34. @Kris: I had a similar question. I made my own gradebook, called ActiveGrade, to handle all of it. It’s definitely time-consuming to enter all of your standards into a computer, but in the end I think it’s worth it – when your gradebook software is set up to handle it.

  35. @Kris: You have to redefine what letter grade your percentages represent. If a 4 is perfect and two 4’s locks in a 5, then the score of a 3 should not be a 60% F. The 60% would represent a C+ or B.

  36. Thanks for the feedback. I’m in a district that insists on all teachers using Power School and District letter grades tied to listed percentages. Looks like I’ll have be a bit more creative to get this brilliant idea working under the constrictions I’m under….

  37. @Kris: I was just using Power School at the school I was at last year. You have the ability to change your grade book percentages under the class options. My AP Calculus class had different percentages than my Algebra 2 class as far as the letter grade went. (Not by much. the AP class could earn an A- if their overall score was at least 89.1%. It was not so in the Algebra 2 class.) But possibly I don’t understand the full scope your constrictions. Also, there should be a Power School contact person or tech person that can help you switch it. I’m using something called Zangle now. Super easy to change the grading scale.

  38. Thanks all. My district has a policy against altering their set grading scale….

    But I’m not giving up and have a few ideas brewing — they’re mainly work arounds. Now I’m wondering about the feasibility of entering so many “scores” for each student. Doesn’t it take an unreasonable amount of time to enter each objective separately (and repeatedly). I love the outcome, but going from entering say 100 quiz grades to 400 objectives for 100 students sounds daunting!

  39. @Kris… I’m struggling with the same thing. But I’ve decided that 3 out of 5 is just going to be a 60%. 4 out of 5 will be 80% and I think that’s what Dan’s intentions were. If they get a 4 out of 5 (because the second question is harder), then a B is reasonable. I think it will be strong motivation for most of the kids to get those 3’s up to 4’s.

  40. Dan, I really like your approach to evaluation. I’m trying to find a way to make it work with our system here in Ontario, Canada. Our grades are broken down as follows:

    70% term work (tests, quizzes, assignments, presentations etc)
    30% course summative/final exam (this may be one or both of these subject to the course, grade and level)

    In addition, the term work is broken into 4 categories:

    Knowledge (for example 25% of the 70%)
    Application (for example 25% of the 70%)
    Communication (for example 10% of the 70%)
    Thinking/Inquiry (for example 10% of the 70%)

    It’s the K/A/C/T part that I can’t figure out how to fit into your system.