Posts

Comments

Get Posts by E-mail

May as well get this out of the way as long as I'm in this public state of contrition.

The concept checklist, in theory, is where students track their progress towards mastery. They write down concept names in rows as we test them and then record their scores (on a four-point scale) along that row, one after the other, each time they retake a concept quiz. I log only their highest scores in the gradebook and whenever they record two perfect scores on a concept, they never have to take that concept again.

The concept checklist is a mess. I run through the same script every year, illustrating the same process with better and more precise visuals every year to no avail. The process confuses students. The process puts students farther from meaningful self-assessment not closer. I saw another checklist crumpled in the trash last week and figured it out.

Their highest score matters much more to me than the specific ordering of low scores preceding it. So forget the earlier low scores. Students add length to the bar as they improve on earlier scores. This checklist design is consistent with our class ethic that "what you know now matters to us more than what you used to know," whereas the other design maintains a permanent record of "what you used to know."

So here's an updated attachment.

BTW: Reader Jacob Morrill does me one (or two or three) better with his adaptation, which is superbly designed:

51 Responses to “On Getting The Concept Checklist Wrong These Last Six Years”

  1. on 19 Jan 2010 at 11:40 pmMr. K.

    I’m stealing this, just like I stole the original.

    But I’m going to split the rows on the score tabs, so they can keep track of two top scores.

  2. on 20 Jan 2010 at 3:21 amBrian

    I like this. It also adds some score-keeping meaning to the stamp, whereas before it was just a [meaningless?] piece of praise to validate what was already written down (I wonder what Alfie Kohn’s thoughts on that would be). Here, if there’s no stamp, there’s still one more to go.

  3. on 20 Jan 2010 at 4:45 amCaitlin

    I’m sharing this with the folks at my school as we discuss keeping track of standards or goals. This seems like a nice way to implement for students.

  4. on 20 Jan 2010 at 5:15 ammr. a

    This is great! I actually use something similar to this with my 4th graders, but you’ve gone up and improved it.

    Going to borrow. Thanks!

  5. on 20 Jan 2010 at 5:50 amAndrew

    I really like the elegance of a “personal best” record that obscures the student’s score. Adding another block to the “meter” means you’ve beat (or tied?) your best work, and it is always good news.

    Is there a level on the score-free chart that denotes mastery? Is three improved scores (reasonable on a 1-4 scoring system), or two perfect scores, or some some other metric entirely. Does reaching mastery have any affect on what they’re asked to do in class?

    As someone who’s mental proceses were warped by video games from a young age, I often try to build class routines that mimic game logic. I’ve used various forms of super meters and XP bars, and this is a nice addition to that toolkit.

    Thanks!

  6. on 20 Jan 2010 at 6:31 amjosh g.

    Ok, that’s funny. I used this for my practicum last fall and assumed it was supposed to be a fill-in-the-boxes template to begin with.

    (Of course, being a practicum, the students were mostly confused by what I was doing anyway as I was changing things up two weeks into the class.)

  7. on 20 Jan 2010 at 7:11 amjacob b.

    Do you no longer require two separate “perfects” in your quiz system?

    Would you track this by adding a 5th column and labeling it “4+” or “5″ or “Heck yea you did it it again!”?

  8. on 20 Jan 2010 at 8:02 amJason Dyer

    Jacob, I would guess the stamp is supposed to be the “perfection” mark (note how the third row doesn’t have a stamp yet, so the student only scored a 4 once).

  9. on 20 Jan 2010 at 8:11 amJason Dyer

    Incidentally, I did try a microassessment system in my AIMS Math class (seniors who haven’t passed our standardized test yet) but the student feedback was overwhelmingly negative. I’m still not sure why, but I think it is partly that they were seniors and set in their ways. In any case I have shifted to a more standard quiz system.

    The other problem with these students in particular is that “what they know now” is a moving target; it’s very hard to presume mastery even after multiple demonstrations.

  10. on 20 Jan 2010 at 8:17 amjosh g.

    Do you still track past scores in your own records? I waffled on that question when I started using this. “More data -> more analysis” is good; but “less data -> less administrative hassle and tidier mark book” is also good. Plus if I really don’t care about past scores then tracking past scores is kind of … weird? Hypocritical? Except not really so whatever.

    (I tried setting up a spreadsheet to get the best of both worlds; got something close enough to use but still felt like I needed to hunt down a few more spreadsheet tricks to make it really click. Or just build a database to do it, maybe.)

  11. on 20 Jan 2010 at 8:26 amCarol

    If I’m following you, shouldn’t your sample student only have one box filled in for concept 4, since the second and third scores are lower than the first score?

  12. on 20 Jan 2010 at 8:50 amElissa

    I don’t get this at all. Why don’t they just write down the dates when they get a 4. If we don’t care about past lower scores, why write them down?

  13. on 20 Jan 2010 at 9:03 amJason Dyer

    Because they might get a 3 and only a 3 even after multiple attempts?

  14. on 20 Jan 2010 at 9:27 amDan Meyer

    Gotta get Jason Dyer a dy/dan merit badge for fielding questions so well. Best I can do on short notice is to urge everyone reading to check out his most recent post outlining a killer intro to area. (Kicking myself for not coming up with the same intro to units.)

    I figured, when I posted this, there’d be some folks like josh g. who saw the original concept checklist, misinterpreted my instructions for it, and used it the right way since day one. Thanks, bud, for letting me wander so far off the trail these last six years.

    Andrew: Is there a level on the score-free chart that denotes mastery? Is three improved scores (reasonable on a 1-4 scoring system), or two perfect scores, or some some other metric entirely. Does reaching mastery have any affect on what they’re asked to do in class?

    Mastery means they get a perfect score twice. Two 4′s. After the first four, there’s nothing more to shade, so on the second four they get a stamp.

    Carol: If I’m following you, shouldn’t your sample student only have one box filled in for concept 4, since the second and third scores are lower than the first score?

    I give them credit for their highest score.

    Perhaps I’ll take this moment to direct other questions to the FAQ.

  15. on 20 Jan 2010 at 11:42 amSarah Cannon

    Seeing the different ways people interpret this is fascinating. I had students write the date in the box under their score. I don’t know why I didn’t switch to the shading in.

    I like Mr. K’s idea best though. Split the rows with a dotted line, easier to visualize the top two scores that are in my gradebook.

  16. on 20 Jan 2010 at 2:40 pmElaine C.

    *heh* I’d interpreted it to mean the shading/date thing from the very beginning. In fact, I’m trying to torture this into one of my Inquiries for BTSA. I *love* the idea of this assessment, and think it’s really something my kids will adore.

    (On a related note… do you ever allow local-ish teachers to observe you for a class or two? And/or do you have any conferences coming up that you’re presenting at? I’m teaching in Northern Monterey County, and would LOVE a chance to see you in action!)

  17. on 20 Jan 2010 at 3:07 pmRiley

    Interestingly, I interpreted this as what you meant all along! My skills checklists have a blank space and I encourage kids to fill them as they will – some fill theirs like a fish tank, others use horizontal bars like you’ve demonstrated above, others use pie-like slices… and some mix and match.

    I’m still working on getting a sweet “master” stamp, though…

  18. on 20 Jan 2010 at 3:37 pmDan Meyer

    A pox on josh g. and now Elaine and Riley. I have to wonder what else of my work y’all have decided to accidentally improve.

    Elaine: On a related note… do you ever allow local-ish teachers to observe you for a class or two? And/or do you have any conferences coming up that you’re presenting at?

    I’ll be speaking at TEDxNYED, which is about as far from local as it comes for us west coasters I’m afraid. I’ll also be doing an online thing on Wednesday, February 17, right in the middle of the school day. I’ll find some way to get both of those online.

    You’re welcome to come up and hang out. I teach mornings on MWF. Just let me know and I’ll hook you up with the requisite badge, security clearance, biometric scan cards, etc.

  19. on 20 Jan 2010 at 4:27 pmElizabeth

    I love this, including all the different improvements being offered. It will be fascinating to see how things play out over time.

    One idea I had that I wanted to propose is — in the spirit of the Interesting Thing Followed By Big Question lesson intros — why not add a “big question”-type of question to the top of the checklist? I can’t put my finger on it yet, but something like, Which skill areas have I mastered?

    Maybe this is just dumb, but I think that a reminder question that is constantly framing the assessment question for the student can help to reinforce the process of self-assessment and lend meaning to the stamp.

    Barf away, if I’m just babbling.

  20. on 20 Jan 2010 at 8:04 pmKevin Young

    I’m intrigued by this and am going to try adapting it to college-level human anatomy labs. One thing I am thinking of doing is once a student has earned a master stamp I will let that student test others and pass on their own stamp of approval. I think testing others will help reinforce what the masters have learned.

  21. on 20 Jan 2010 at 9:24 pmDan Meyer

    Brave. I like it.

  22. on 20 Jan 2010 at 9:40 pmJYB

    I use a bar graph system. They graph their scores along the x-axis on a 0-4 scale.

    http://www.scribd.com/doc/25519003/Tracking-Forces

    The big black square is the graph, it didn’t really come out well on scribd.

    On the back they just traffic light the specific standards (check/plus/minus).

    I think “what you used to know” is valuable especially for the students who are behind and only compare themselves to their peers. I spend a lot of time pointing out the first few scores to the students to point out how far they’ve come. Also I think it helps to have to point out that everyone started low, the “smart kids” didn’t start out with 4s.

    I’m curious about your reasoning for overwriting that record.

  23. on 21 Jan 2010 at 6:48 amJason Dyer

    JYB, I think the line ‘The process confuses students. The process puts students farther from meaningful self-assessment not closer. I saw another checklist crumpled in the trash last week and figured it out.’ makes it fairly clear the more extensive record was too confusing to the students to be useful.

    This sort of thing I’ve found is dependent on incoming student background. I’ve had classes that could handle an even more complicated version of Dan’s original form, and I have had one class for which even the above simplified version would be too confusing for them.

  24. on 21 Jan 2010 at 9:42 amDan Meyer
    Jason: … the more extensive record was too confusing to the students to be useful.

    This is exactly right. While I prefer more data to less, that extra data comes at an organizational cost. JYB has something pretty cool running there with what appears to be two pages per concept and a separate rubric for each.

    I can count on my students to hold onto exactly one piece of paper — their opener — and that only for two weeks at a time. I have to watch my overhead.

  25. on 21 Jan 2010 at 11:55 amSam

    This is so interesting. I too didn’t fully read the original instructions, and figured if I am only keeping track of the highest score, then so should the students. They were erasing and re-writing checkmarks, until one of them came up with the bar graph system! I will make sure to tell him that you approve of the idea.

    Also, as long as I am giving feedback on the system, my students LOVED it. It is an 8th grade Algebra class (I also teach Geometry), and these kids were so excited.

    I am attempting to adapt the system for a Biology class, and as you said originally, the trick is in defining what is a concept.

  26. on 21 Jan 2010 at 12:09 pmElissa

    Yes, can we have another post on how to decide concepts and design assessments? I’ve read the comprehensive resource a hundred times but I still need more.

  27. on 21 Jan 2010 at 5:55 pmDan Meyer

    @Elissa, the concept list is one of the hardest, most fun, and most personal aspects of this kind of assessment design. I don’t know what else I can say on the matter but you might enjoy connecting with Riley Lark, who asking a lot of the same questions that you are.

  28. on 21 Jan 2010 at 7:52 pmJYB

    I get what you’re saying about confusing them. The topic scales I use are for everything they do, including projects, so we get pretty used to them. In the beginning of the year though it’s just crazy.

    Your checklist just screams sparklines to me. A little mini line graph running to the side.

    Or you could drop a couple extra grid lines across where the score would go for series of bar graphs. Then they could just shade them in.

    I’ve found the added value of seeing visual growth is worth the extra hassle.

  29. on 21 Jan 2010 at 7:55 pmElissa

    Yeah I’ve been reading his stuff but I still don’t know how to get away from assessing everything. Why do we teach it if we don’t assess it?

    Of course, at this stage, I’m wondering why I teach anything I teach.

    How do I get students to buy in if there is no grade attached? They don’t know the value of learning for the sake of learning.

  30. on 26 Jan 2010 at 8:22 pmAlex

    Feel I owe you some royalties. Been using the comprehensive assessment all year, the kids like it and it’s helped me a ton. A friend/fellow teacher in my district is also using it, and she shared it at the district level. The bigwigs liked it, and I’ve received complements as a result. So, my friend, complements forwarded to you.

  31. on 29 Jan 2010 at 5:08 amKendall

    I have never used a concept checklist before, and am interested in how you format & grade assessments so that students know how to fill in the checklist.

    * Do you label each question/task with the concept it is covering?
    * Do you grade all questions/tasks on a 4-point scale?
    * Does every assessment cover only one concept?

    I guess I just don’t see how you implement this so that you don’t need to help every student fill in their checklist.

  32. [...] me, @dy/dan’s post “On Getting The Concept Checklist Wrong These Last Six Years” was a part of key that unlocked things for me and made it all seem possible. The pictures [...]

  33. on 27 Dec 2010 at 10:05 amDaniel Schaben

    Thank you for posting this idea. I revamped my entire assessment strategy with my students. They love it and I love it. I have students motivated to take tests. In fact I have had several students that normally would not have come in for extra help come in. It has also completely organized my curriculum. There is no more guessing on what you need to study to do well in my class. About the only change I made was making the rubric a five point rubric instead of four to match my style of grading. Again thanks for the ideas. This site is a gold mine.

  34. [...] You ripped off the idea for those little squares on the skills sheet, and you’re going to have a bad day when someone’s score goes down after they’ve [...]

  35. on 22 Feb 2011 at 1:26 pmRoberto Marquez

    Just started using this in the second semester. After first quiz, I like that I can break down how well class does (or individual is doing) at each concept

  36. on 22 Jul 2011 at 9:46 amKevin Lade

    This is an interesting thread that matches an issue that my school has just been discussing. I’m not sure if anyone is following the thread anymore since it’s been several months since the last comment, but I’ll post a comment anyway.

    At the start of a unit I provide to students the rubrics for whatever learning goals are in the unit. The rubric for a learning goal easily fits on the front of a sheet of paper and on the back of that sheet is a form for graphing scores. Actually, there’s more on the form than this. There is a spot to record the first score and a spot to record what the student’s goal is by whatever date. There is a large area to write out an action plan of what the student will do to increase performance (e.g., complete more practice, study with a friend, whatever). Below this is an area to graph the scores: the x-axis is the # of the assessment and the y-axis is the score from 0 to 4. The actual graphing could be a bar graph or line graph (I use a line graph because it better fits my Algebra 1 topic of scatterplots and regression).

    Now, this seems to be the opposite of what folks are discussing here, but the reason I like it is because the student can graphically see a trend, whether there’s steady improvement, or stagnant scores, or even if a score dips down. Following each assessment the student plots the next score on this graph and adjusts the action plan accordingly (or maybe finally commits to their first action plan). To me a process such as this is what makes an assessment truly formative for a student–he uses the feedback from an assessment to make and change what he’s doing to understand the material.

    Nothing about a system that maintains a record of only the highest score achieved precludes the response to feedback procedure that I’ve described, but the visual depiction of a trend would seem to have more power to influence a student’s habits.

    But, perhaps I’m totally wrong on this and your system is superior. Certainly, your system has a compactness to it of putting all learning goals on one sheet of paper. In my system each goal’s rubric/graph is on its own sheet of paper. My gradebook software does print a report of all of the learning goals; I’m only talking here of the manual monitoring of learning that the students themselves are doing on paper.

  37. on 23 Jul 2011 at 12:45 pmDan Meyer
    Kevin: Nothing about a system that maintains a record of only the highest score achieved precludes the response to feedback procedure that I’ve described, but the visual depiction of a trend would seem to have more power to influence a student’s habits.

    Love the graphical approach. Obviously the downside is more paper, but the upside is considerable. Thanks for sharing.

  38. [...] system, that every student knows exactly what she does and doesn't know. [2011 Aug 26: here are my most recent thoughts on the concept [...]

  39. on 20 Sep 2011 at 1:40 pmKris Kramer

    Anyone using PowerSchool with this? Any advise? If I give 3 points out of 5 that’s a 60%. Doesn’t sound like what we’re going for. Also, is partial credit out the window? Finally, I can’t help but wonder if maintaining records and logging a bunch a scores for each test/assessment doesn’t take an insane amount of time. May sound selfish, but I’m attempting to be realistic.

  40. on 20 Sep 2011 at 3:45 pmRiley

    @Kris: I had a similar question. I made my own gradebook, called ActiveGrade, to handle all of it. It’s definitely time-consuming to enter all of your standards into a computer, but in the end I think it’s worth it – when your gradebook software is set up to handle it.

  41. on 20 Sep 2011 at 11:03 pmIan

    @Kris: You have to redefine what letter grade your percentages represent. If a 4 is perfect and two 4′s locks in a 5, then the score of a 3 should not be a 60% F. The 60% would represent a C+ or B.

  42. on 21 Sep 2011 at 2:26 pmKris Kramer

    Thanks for the feedback. I’m in a district that insists on all teachers using Power School and District letter grades tied to listed percentages. Looks like I’ll have be a bit more creative to get this brilliant idea working under the constrictions I’m under….

  43. on 21 Sep 2011 at 3:43 pmIan

    @Kris: I was just using Power School at the school I was at last year. You have the ability to change your grade book percentages under the class options. My AP Calculus class had different percentages than my Algebra 2 class as far as the letter grade went. (Not by much. the AP class could earn an A- if their overall score was at least 89.1%. It was not so in the Algebra 2 class.) But possibly I don’t understand the full scope your constrictions. Also, there should be a Power School contact person or tech person that can help you switch it. I’m using something called Zangle now. Super easy to change the grading scale.

  44. on 05 Oct 2011 at 11:45 amKris Kramer

    Thanks all. My district has a policy against altering their set grading scale….

    But I’m not giving up and have a few ideas brewing — they’re mainly work arounds. Now I’m wondering about the feasibility of entering so many “scores” for each student. Doesn’t it take an unreasonable amount of time to enter each objective separately (and repeatedly). I love the outcome, but going from entering say 100 quiz grades to 400 objectives for 100 students sounds daunting!

  45. on 26 Oct 2011 at 9:55 amEric

    @Kris… I’m struggling with the same thing. But I’ve decided that 3 out of 5 is just going to be a 60%. 4 out of 5 will be 80% and I think that’s what Dan’s intentions were. If they get a 4 out of 5 (because the second question is harder), then a B is reasonable. I think it will be strong motivation for most of the kids to get those 3′s up to 4′s.

  46. [...] And then I explained my SBG system. [...]

  47. on 06 May 2012 at 4:24 pmKate

    Dan, I really like your approach to evaluation. I’m trying to find a way to make it work with our system here in Ontario, Canada. Our grades are broken down as follows:

    70% term work (tests, quizzes, assignments, presentations etc)
    30% course summative/final exam (this may be one or both of these subject to the course, grade and level)

    In addition, the term work is broken into 4 categories:

    Knowledge (for example 25% of the 70%)
    Application (for example 25% of the 70%)
    Communication (for example 10% of the 70%)
    Thinking/Inquiry (for example 10% of the 70%)

    It’s the K/A/C/T part that I can’t figure out how to fit into your system.

  48. [...] present on Understanding by Design. Dan Meyer explored the idea quite a bit using his term of the concept checklist. Shawn Cornally talks on his blog about really pushing the idea to give students the freedom to [...]

  49. on 09 Sep 2012 at 1:52 pmWhy Give Homework? | Mr. V's Class

    [...] of the “flipped classes” and “mastery learning” and “independent practice” all center around one thing: Learning [...]

  50. [...] I created it in Pages, the idea was stolen from Dan Meyer and others.  The circles are for “stop-lighting” their progress on the learning [...]

  51. on 04 Nov 2012 at 8:29 pmWhat works so far « multiplayermath

    [...] each day and see the results of their completion of assignments and successful mastery of concept quizzes. When they reach certain milestones and level up, they get an item for their avatar and they can [...]