Winter Quarter Wrap-Up / Spring Quarter Kick-Off

Brief Remarks Encapsulating Winter Quarter

  • Mentorship. This is new: I switched emphases from teacher education to math education. I’m retaining Pam Grossman (my current adviser in teacher education) but adding Jo Boaler (who is the math education professor at Stanford) to the Team Dan Meyer, Ph.D. roster. The education of new teachers and development of current teachers is still wildly fascinating to me, but I am asked with growing frequency to speak to and write for and work with math educators. I know enough about what I don’t know to know that I need to study up and work out some blind spots in my vision if I’m going to be effective in any of those roles.
  • Temptation. The private sector extended several invitations my way last quarter to leave Stanford – to cut a corner, basically, and go straight to work. Some of those invitations were easier to turn down than others. In every case, though, I was grateful for the opportunity to remind myself again of the reasons I committed to this difficult, frequently humbling work.
  • Music. I tend to wear out the grooves on a single record during finals week each quarter, playing the same songs over and over and over until they become useful white noise. Fall quarter it was Mumford and Sons. Winter quarter it was the soundtrack to The Social Network by Trent Reznor and Atticus Ross. Anyway.

Notes on last quarter’s classes:

  • Statistical Methods in Education. Key skill: analyze regression tables like this one for meaning. Prof. Stevens said in fall quarter he loves the moment when an author drops the tables in a paper because up until that point we’re just bobbing along with the author’s narrative. But the table tells its own stories.
  • Proseminar. One of my colleagues said it pretty well: “In any given week of proseminar, two thirds of the class simply don’t give a damn.” Which is to say the wonks don’t really care much about the pedagogy and the teachers don’t care much for policy and the social theorists have an entirely separate set of interests.
  • Casual Learning Technologies. This was a mixed bag. The field is really, really new (James Gee, the discipline’s flag-bearer, is a linguist by training who got interested in gaming all of six years ago) and has a lot of room to grow. Which is to say, I wasn’t dazzled by the literature. Remind me to post my group’s final project, though. That was fun.

Current Coursework

  • EDUC325C – Proseminar. David Labaree, Francisco Ramirez. Required. Labaree, in his initial remarks to the class: “You may have heard this course features too much reading, too much writing, that the criticism is too harsh, and our opinion of schools is too pessimistic. It’s all true.” (Labaree has written a few books of note.)
  • EDUC359F – Research in Mathematics Education. Jo Boaler. Elective.
  • EDUC424 – Introduction to Research in Curriculum and Teacher Education. Hilda Borko. Required.

Winter Quarter #GradSkool Tweets

  • Yes, this is #gradskool and, yes, Angry Birds is on the syllabus. 6 Jan
  • Today’s #gradskool throw-down: Who won in US schools and universities — Dewey or Thorndike? Great discussion. Lots of nuance. 18 Jan
  • Stats prof, reading the room: “I don’t know how to make this more lively. I really don’t know how to make this more lively.” #gradskool 23 Feb
  • Carol Dweck is speaking. I am listening. #gradskool 8 Mar
  • Dweck has no slides. She’s four-feet tall, sitting on a table, feet dangling beneath her, positively /owning/ the room. #gradskool 8 Mar
  • Five rows from Michelle Rhee. An unlikely mix of education and business grad students in the building. 11 Mar
  • Rhee: “What we did definitely made people unhappy.” She literally seems to believe that diplomacy and efficacy are mutually exclusive. 11 Mar
  • Rhee: “Is there a less controversial way to do controversial things? I don’t know the answer to that.” 11 Mar
  • Rhee: “Chris Christie? I love him. He’s a Republican and I’m a Democrat. It’s not obvious we’d get along so well.” Seriously? 11 Mar
  • Rhee: “I worry about people going into the job with longevity as one of the goals. I’m not a big believer in longevity.” 11 Mar
  • GSB student: “Did you really eat a bee?” Rhee: “I did eat a bee.” Way to pitch her a fastball, Chuck. 11 Mar
  • These moguls were the most out of place contingent at the Rhee Q&A. Good luck finding the executive washroom, fellas. 11 Mar

Michelle Rhee followed me on Twitter the next day. So look out, right?

Favorite Winter Quarter Papers

I spent a few weeks of my winter quarter trying to make sense of the PBL / anti-PBL scrum of 06/07. Those papers are below, in chronological order, with a closing paper pitched specifically at math educators.

Spring Speaking & Workshops

Tagged in:,
I'm Dan and this is my blog. I'm a former high school math teacher and current head of teaching at Desmos. He / him. More here.


  1. Dan-
    Big fan of your work. Just curious if you have seen “teach like a champion” by Doug Lemov? Really solid teacher education material, looks to be right up your alley, I was curious what your take on it is.

    I teach in Ny state where we are going through a ton of serious value-added related changes. I’m working on a committee for new annual professional performance review (Appr), and was wondering what materials you would recommend to new teachers who need direction/improvement.

    Keep up the good work, your stuff is always interesting and inspiring-
    Doug Cochran

  2. So happy to see the favorite papers section again — loved it last time, delighted to have a handy reading list ready for the weekend!

    Your tweets about Rhee…go a long way towards not changing my opinion of her. ;-/

  3. As a reader of your blog for the past nine months, I need to say “Thank you, thank you, thank you, AND thank you.” or more precisely, “Thank you for switching to Mathematics Education, thank you for working with Jo Boaler, thank you for not taking the short cut and bailing, AND thank you for publishing the reading list!” Maybe one more, “Thanks for having such energy and sharing.”

  4. Love that you peel the curtain back and give us a peek at what you’re up to. I find it fascinating. Thanks.

  5. Thanks a lot for your papers section, I love it. Your blog is one of my favorites. I’m doing my PhD at Montreal. It would be nice to be in yours shoes for your Boaler’s course (EDUC359F). If you can send me some informations, I’ll be happy :)

  6. I’d love to read the papers you listed re the PBL/anit-PBL scrum. I’m a first year teacher which means I’m trying to figure out whether and how to balance inquiry vs. practicing techniques. It also means I’m not reading those papers until July, if then. Since you are my go-to inquiry hero, throw me a rope: when if ever should one teach algorithms? Did those papers change your thinking? How did they, or why didn’t they?

  7. I’ll echo Dan’s question. I read the last paper first and was fascinated by the chess analogy. I’d love to hear your thoughts, but maybe you’re waiting for us to form our own opinions first?

  8. @ Stacy (cc Dan and Dan) –

    I just read the last paper too. Looking forward to Dan M’s thoughts. My 2 cents is that Sweller et. al. have a really thin understanding of learning; they’re reducing the whole question to whether the objective is “problem-solving strategies” or “specific mathematical methods,” as though the point of inquiry learning (actually, I don’t like this phrase because it’s so buzzwordy; I need another shorthand name for this – maybe “letting kids effing think for themselves for a minute,” courtesy Shawn?) is just to replace instruction in the latter with instruction in the former. Speaking as a former chessplayer, of course it’s true that learning how to play chess means learning a lot about specific kinds of positions, what works and doesn’t work in those positions, etc., and that studying games well-played is a great way to do this. I learned a huge amount by studying well-played games. But they must be crazy if they think that this process is going to happen if you are not also playing a lot of effing chess (i.e. doing tons of no-guidance problem solving) before, during and after your study of great games. Without the context of playing the game all the time, I wouldn’t have cared enough in the first place to dig into those well-played games, I wouldn’t have known what lessons to take from them, how narrow or broad the context in which those lessons applied, and I wouldn’t have appreciated the solutions I saw. More broadly, if I had not been constantly trying to solve my own chess problems (in the context of playing games, where the effectiveness of my problem solving determined whether I won or lost and even more so determined whether I was happy with my play), the study of good solutions to other people’s chess problems would have been pointless and deadening and I would never have done it, and if somebody had made me do it I would have been miserable and retained nothing.

    Okay that was about 8 cents.

  9. @Ben, Stacy, and Dan,

    Here are a few of my notes:

    Sweller et al: There is no body of research based on randomized, controlled experiments indicating that such teaching leads to better problem solving (p. 1303).

    Their insistence on randomized, controlled trials is admirable, but the reason you don’t see more of that kind of research design in education isn’t because ed researchers are lazy or unserious or because they haven’t been forced to take hundreds of seat-hours of stat methods courses in grad school. It’s because the classroom environment is really hard to control. We aren’t testing the effectiveness of aspirin over a two-hour release. We’re testing the effectiveness of curricula and instruction over a year. That’s messy business. Look no further than their own research to see how much you have to distort what constitutes curricula and instruction to manage a randomized, controlled trial.

    So they insulate themselves from reams of conflicting (if not fully contradictory) research by insisting on a standard that they themselves can’t convincingly clear. That doesn’t impress me.

    For instance, I just read Benny’s Conceptions which is a lock for one of this quarter’s favorite papers. It argues convincingly that the use of worked examples over four years of a student’s education has had profoundly negative effects on his conception of mathematics. I wouldn’t say this sticks the knife in worked examples as a valuable instructional strategy by any means. I’m saying that Sweller et al. won’t even bother themselves to address the implications of this kind of study because it doesn’t meet their rigidly defined qualifications for participation in the debate.

    That’s unimpressive.

    Worked examples may be the most effective method delivering short-term gains in the instruction of a certain kind of problem. (They tested symbolic manipulation in their original study, immediately after instruction.) But damn. The practical implications are huge. How do you manage to motivate students over a year to submit to that kind of instruction, which Sweller et al. admit they’ll probably find miserable? Are worked examples effective for every kind of problem? Do I need a separate worked example when solving for distance, rate, and time? I’m genuinely curious about implementation, but their response to the issue of motivation (kids will be motivated to submit to worked-example instruction because they’ll be working through more examples later) indicates they’re deeply unserious about the practical implementation of their preferred approach.

  10. One issue with PBL scrums (and this happens at both ends) is they often go down with a philosophical background at contention rather than a utilitarian outlook. This tends to make the landscape a war fought on a continuum rather than the more complicated topology complex systems tend to.

    This can lead to (for example) the simplification that all problem-based learning is conceptually the same, when there are multiple approaches and contexts with significant differences where some criticisms may apply and some may not. (Specific example: the Kirchner et. al. mentions the criticism that PBL methods can lead to a difficulty in generalizing and that students trying to explain science were only able to explain them in the context of specific cases. How does this account for, say a PBL approach to geometry proofs where each specific problem/proof *is* the general thing being taught?)

  11. Jason: This can lead to (for example) the simplification that all problem-based learning is conceptually the same, when there are multiple approaches and contexts with significant differences where some criticisms may apply and some may not.

    That’s the objection raised by Hmelo-Silver et al but Kirschner et al specifically address it:

    Furthermore, while scaffolding, like all guidance for novices, is better than no scaffolding, the ultimate scaffold, providing learners with all information needed including a complete problem solution–either prior to a task or just-in-time during a task–is better still (2007, p. 117)

    Basically, they acknowledge there are many different forms and degrees of PBL but then note that all of them are the same in offering something less than 100% guided instruction through worked examples. Which is most effective. Therefore all forms of PBL are less effective.

  12. While not part of “spring” so doesn’t fit your spring conferences list, I am so excited about the workshop that you are going to part of in Richmond, VA in late June.

    I will be there with all the giddiness of a screaming fan at a rock concert, where you are the rockstar. Thank you for your blog and challenging the status quo. You inspire me with you dedication and innovation.

    Keep up the great work.

  13. asically, they acknowledge there are many different forms and degrees of PBL but then note that all of them are the same in offering something less than 100% guided instruction through worked examples. Which is most effective. Therefore all forms of PBL are less effective.

    Ayep. I would still qualify Kirschner and Sweller’s “awareness” of PBL variations in air quotes.

  14. Does the chess master gain his extensive memory of the chess board by studying worked examples? He likely studied plenty of famous games if he’s a real enthusiast. But by and large, i’m sure his knowledge came from actually playing chess, coming onto difficult problems, solving them, and analyzing the results afterwards. The chess master didn’t need anyone to systematically show him thousands of games that highlight different board combinations. Chess players learn from plenty of different sources, but i’m sure even Sweller would agree that he memorized so many board combinations by playing lots and lots of chess!

    It would be interesting to see a study comparing student engagement or long-term results in PBL versus some kind of worked-problem curriculum.

  15. Alex: It would be interesting to see a study comparing student engagement or long-term results in PBL versus some kind of worked-problem curriculum.

    I agree. That kind of study is unlikely to be randomized or controlled, though, so don’t expect it from Sweller et al, no matter how essential it is to the matter at hand.

  16. How do you manage to motivate students over a year to submit to that kind of instruction, which Sweller et al. admit they’ll probably find miserable?

    Sadly, in my experience, students subjected to that kind of instruction for long enough eventually acclimate to such a degree that any other approach to learning mathematics (to use those two words loosely) is unthinkable. While they may not enjoy that kind of instruction, they become so comfortable with it that they vehemently object to anything else.

    And then there are the students who actually do enjoy that kind of instruction, and do well under it, and believing they are “good at math,” while they may really just be good at cataloging and following directions.

  17. From the value-added paper (Glazerman et al.) Dan posted:

    ‘We do not advocate using value-added measures alone when making decisions about hiring, firing, tenure, compensation, placement, or developing teachers, but surely value-added information ought to be in the mix given the empirical evidence that it predicts more about what students will learn from the teachers to which they are assigned than any other source of information.’

    This is a striking conclusion, and one that should reframe the debate about this measure amongst teachers. The stigma attached to value-added- a heartless measure that doesn’t take into account the ‘art’ of teaching- is methodically (almost politely) dismantled by Glazerman and his people. One of those sharp and lean arguments where you can go in thinking one thing and come out shaking your head.

    It appeals to those interested in policy, for obvious reasons, but also to fans of statistics. The comparison between SAT and batting averages was fascinating.

    Coupled with the ‘moral conversation’ reading (the only two I’ve gotten to thus far), this is already an outstanding list. DM make this a bi-weekly or monthly feature, if you have time. Thanks man.

  18. @Alex re: “long term PBL study”

    Hattie, in Visible Learning, points to a study showing increase in “self-directed learning (d=0.54) and attitude toward learning (d=0.52).

    The catch is its an unpublished study.

    Smith, R. A. (2003) Problem-based versus lecture-based medical teaching and learning: A meta-analysis of cognitive and non-cognitive outcomes. Unpublished Ph.D., University of Florida

  19. I am not sure I agree with the SAT and baseball player analogies for value added measures of teacher performance.

    Batting averages change a lot from year to year because player performance changes a lot from year to year – injury, age, steroids, and confidence make a big difference from year to year. Value added scores should not change a lot from year to year because teaching performance does not change a lot from year to year. I am pretty much the same teacher this year as I was last year. Batting averages reflect actual changes in performance. Value added scores mostly reflect random factors, not teaching performance.

    On the SAT, a student will generally get very similar scores on multiple retakes of the SAT, so it is a consistent test. It is a crude, but consistent measure of a narrow sort of ability. That is why the colleges use it. It is true that it doesn’t predict GPA very well, but that isn’t what it is used for. Value added will be used as a measure of teacher effectiveness even though it is not reliable. SAT is not used to predict student GPA in college, so it doesn’t matter that it does a poor job of it.