This is ten months of grad school in ten minutes:
That video has me explaining the research from my qualifying paper, which is the culmination of a grad student’s second year at Stanford’s School of Education. It qualifies you, in the eyes of your advisers, to take on the much larger research project they call a dissertation.
The Experiment.
I showed two groups of students an image of a water tank. One group saw all the information and abstraction relevant to the question, “How long will it take to fill?” The other group just saw the question, “How long will it take to fill?” and had to request the information and develop the abstractions themselves. If you’re remotely aware of this blog’s obsessions, you can guess the research questions I asked about that experiment. (Watch the video!) Perhaps the most surprising outcome of the experiment (to me) was that the higher-achieving math students in the study really disliked not having all the information and abstractions in front of them.
If you’d like to read the paper, you can feel free. If you have some commentary or criticism that’d profit us here, you’re welcome to the comments.
A few other notes about the qualifying paper, my second year of grad school, and my next year of grad school:
On criticism.
Stanford gives great feedback. The school of education has several schools within it. My school, Curriculum and Teacher Education, does a great job preparing its students for the qualifying paper. In the spring of your first year, you take an introductory course. In the fall of your second year, you take a doctoral seminar that builds to a proposal for the qualifier. In my particular case, I had a qualifying committee that was generous with feedback when I needed it. I also developed the study while taking a course with Alan Schoenfeld at UC Berkeley. So my ideas and writing had as many as eight sets of eyes on them, as needed. (And that’s just faculty. My student-friends gave great feedback also.) That’s amazing and, from my understanding, kind of rare in doctoral programs. That criticism was occasionally contradictory, however, which required a certain discernment I haven’t really developed yet.
The criticism I remember most vividly: a) my weak review of the literature, b) the sense that I wasn’t really taking myself anywhere new with the study, and c) a claim about equity that had me reaching beyond my data.
Great classes I took.
- Accelerated programming in C++. I had no business in an accelerated course in anything related to programming but I had a scheduling conflict and they weren’t putting the standard class online. It nearly ate me alive but spat me out a better programmer and granted me a great deal of sympathy for students who felt like idiots in classes that I taught.
- Analysis of Social Interaction. With Ray McDermott, if that name means anything to you. If it doesn’t, read “Can We Afford Theories of Learning?“, which begins, “If American culture were an Internet, the domain name “learning” would be owned outright by the testing services that use it to feed the yearnings of parents and their schoolchildren.” So a great quarter, basically.
- Front-End Programming in Javascript, HTML, and CSS, another course I took from afar and watched online. Patrick Young is one of the best lecturers I’ve had at Stanford and certainly the best I’ve had in the CS department. Really an invaluable course. I called my final project, “Better Online Math” and it’s the closest thing I have to a dissertation proposal.
- Qualitative Analysis, with Pam Grossman (one of my advisers) and Sam Wineburg, who have taught together for decades, dating back to their time together at the University of Washington. Put these items under the heading “exceeded expectations”: a) the four assignments, b) the syllabus, c) their respect for our time.
- Directed Research, for practical reasons. Make sure you write that one down, class of 2016-2017. There’s no excuse not to max out your units.
Papers I flagged as being particularly worthwhile.
- Berman, P. & McLaughlin, M. (1979). An exploratory study of school district adaptation.
- Brown, JS. & Burton, RR. (1978). Diagnostic models for procedural bugs in basic mathematical skills.
- Cohen, D. (1990). A revolution in one classroom: The case of Mrs. Oublier.
- Egan, K. (1999). Education’s three old ideas, and a better idea.
- Geertz, C. (1972). Deep play: notes on the Balinese cockfight.
- Lareau, A. (2000). Common problems in field work: A personal essay.
- Pollak, H., Albers, D. & Thibodeux, M. (1984). A conversation with Henry Pollak.
- Schoenfeld, A. (2011). A modest proposal.
- Schoenfeld, A. (1998). Making mathematics and making pasta: From cookbook procedures to really cooking.
- Small, M. (2009). ‘How many cases do I need?’: On science and the logic of case selection in field-based research.
- Tyler, R. (1950). Basic principles of curriculum and instruction.
What I’ll be doing my third year.
I have this image in my head from a movie from my childhood that I’ve forgotten. A man stands with one foot on each of two rowboats that are side-by-side. It seems like a good, fun idea at first but then the boats start to drift apart. His weight bears down on both boats, pushing them farther and faster apart until he falls in the water and we laugh.
One boat is christened “Grad School” and the other is “The Other Stuff.” The thing I can do to help myself right here is tie several cords from one to the other, committing myself to projects, papers, and talks that are researchable or that will, at least, inform my research. I just don’t have the time for a long stay in grad school but I may not have the skill to get a dissertation done quickly either.
So for the third year:
- I’m still designing tasks for and consulting with publishers in the US and elsewhere.
- I’ll be facilitating some workshops and speaking at some conferences.
- I’ll be taking the winter quarter off to work with The Shell Centre in the UK. (Did you guys know they pilot their tasks five times before they release them. What new questions do they ask in each new pilot? Let’s find out this winter, okay?)
- I’d like to submit a dissertation proposal at the end of this school year. Vegas oddsmakers are frowning at that one, though.
- I’ve taken the required major coursework for the education doctorate but I need to complete several more courses in my minor emphasis in computer science. I’ll be taking as many of those as I can this next year, online as much as possible. As I narrow in on a proposal, I’ll take some appropriate methods courses also. (ie. if I plan to run an experiment, then something in experimental methods.)
- I’ll continue to develop 101questions into the tool I need to be.
Of course, all of this has been and will be more fun with you guys tagging along, chirping comments and critiques at me as we go.
2012 Sep 13. Elaine Watson posts some thoughtful commentary on my qualifier.
Featured Comment
What would you do with a doctorate degree that you are not already doing?
20 Comments
Michael P
September 11, 2012 - 7:51 pm -At one point in the paper you say that your ability to accurately assess the power-solving prowess of a student gets distorted by the inclusion of the full text. The implicit suggestion to test-makers is that if you want to assess, say, a student’s ability to make sense of a problem and persevere in solving a problem, then you might need a question with less text.
Creating a paradigm for accurately assessing problem-solving ability might be a good career move, I imagine.
Is there a way to conceptually distinguish between the type of confidence that comes from having a simpler, pre-chunked problem and the kind of confidence that comes from having robust knowledge?
What would sort of evidence would weigh against your ideas? Certainly if kids reported being more interested and having more fun with the prefab problems. What about the claim that having kids come up with strategies without the help of a text improves problem-solving skills? I suppose if you developed a reliable measure of problem-solving prowess then you could put various instructional strategies to the test.
Interesting stuff, as always.
Raymond Johnson
September 11, 2012 - 8:37 pm -As someone in a similar spot on their path to a PhD, I’m always fascinated by comparisons with other programs. We take all of our core methods courses as a 1st year cohort (which gives us a common, strong foundation for research) while you mix them with other courses of interest (which gives you some freedom for taking what you need when you think you need it). I know these types of posts might not be appealing to the broadest of educational audiences, but for my own personal interests I hope you keep writing them. I’ll be doing the same.
Bruce James
September 11, 2012 - 9:36 pm -What would you do with a doctorate degree that you are not already doing?
Marti Canipe
September 11, 2012 - 10:15 pm -Like Raymond, I am fascinated by the ways that different programs are structured. I just began my second year of my PhD program and passed my qualifying exam last week. In our program it is basically a defense of a paper written in your first year that is reflective of your research interests which is then used as a springboard for a discussion of what path you envision for yourself through the rest of the program and beyond. This discussion leads to the hammering out of the plan of study for the remainder of the required coursework.
Julia Tsygan
September 11, 2012 - 11:14 pm -dy/dan @ the shell centre, huh? Are you sure you’ll be able to leave once you get there? Seems like a perfect match to me.
Fawn Nguyen
September 11, 2012 - 11:48 pm -“Perhaps the most surprising outcome of the experiment (to me) was that the higher-achieving math students in the study really disliked not having all the information and abstractions in front of them.”
I’m a little bit sad when I read this because unfortunately I’m not suprised as I see the exact same thing in my “high-achieving” middle school math students. The ones who do well on tests do well because tests are written with information baked into them. These kids know that and feel safe when covered under the blanket of formulas and numbers. It’s their comfort zone. But when we do problem-solving tasks (and we do many), including 3-Act lessons, these same high flyer test takers tend to be more reserved, more annoyed and impatient, than their peers who get Cs and lower on tests.
Mainly this says that my tests suck in measuring problem-solving skills. Like Michael mentions above, a test to measure the students’ embodiment of mathematical practice 1 (make sense and persevere) should have less text yet remains rich, interesting, and has multiple strategies. Ironically, we know kids hate “word” problems, yet textbooks keep giving them wordy math problems that make English learners wish they’d signed up to study Russian instead, or at best, they bore the crap out of them. My biggest joy in a 3-Act lesson is waiting for students to ask me what information they need in Act 2. In some instances, I’ve said to them, “You may only ask for two pieces of information, ask wisely.”
I’m wondering if you’d see a bigger difference in confidence levels had you done the experiment with high school or even 8th graders. Also, about the criticism of your weak review of literature, is there much out there to review? Your blog followers’ blogs, albeit self-published, should count for something. Your concept of 3-Act lessons and generous sharing of them have done more for my teaching and my students’ learning than what any textbook has done.
Elena Tomasetti
September 12, 2012 - 6:26 am -Whatever workshop you’re fascilitating, I want to be there. This is my 13th year teaching mathematics and I’ve never enjoyed it so much! My students are having fun and learning too! Thank you for all that you share with us and creating the forum for other good teachers to share their ideas. You’ve inspired me to become the teacher I always wanted to be! Thank you.
Belinda Thompson
September 12, 2012 - 6:42 am -Thanks for sharing your paper and plans.
Regarding PhD programs: I’m in one now, and I entered wanting to learn how to help teachers and students. I was saddened by the realization that teachers are not typically the audience for this work, and that rather, it’s other researchers. In fact, teachers have little or no access to the work. I was a teacher for several years, and have now read published research that would have been extremely useful to me had it been available (shucks, if I had even known that it existed!) I’m continually confused by this system.
To Fawn #6: I had the same experience with my sixth graders. In my “honors” class, there was a clear distinction between the memorizers and the meaning-makers. I worked hard to emphasize meaning-making, but it was sometimes a tough sell to the students and their parents. Stay strong.
Dan Meyer
September 12, 2012 - 9:27 am -@Michael P, good questions. I’m aware of various instruments for assessing problem solving abilities, some designed by AH Schoenfeld. In general, “problem solving” seems as hard to pin down as “transfer,” which makes it an interesting task. For my small paper here, I found myself observing the presence of “not problem solving” which still doesn’t put me any closer to measuring “problem solving.”
@Raymond, thanks for passing on the link to your own reflections. I’m not sure how I haven’t subscribed to your blog until now. The RYSK tag is great.
Bruce James:
Three things:
@Julia, I’ll get everybody autographs.
Fawn Nguyen:
Fair questions. The fact is that I conducted this research at a selective university that enrolls students who have done very, very well doing school. To the extent that doing school means memorizing and applying formulas to contexts that are fully abstracted, their hostility toward the experimental treatment (no data) makes sense.
There’s nothing with “Three Acts” in the title, but that’s not really what I need anyway. There’s literature on the use of multimedia, on abstraction, and on problem solving, and I should have reviewed that more thoroughly.
Yaacov Iland
September 12, 2012 - 3:24 pm -<3 !
Watching this made me ridiculously happy.
It's so clear from the clips that these people are enjoying the challenge.
Jim P
September 12, 2012 - 4:49 pm -Thanks Dan. I’ve flagged this (and the papers you’ve listed) as one to go back to later as I am also going through grad studies. Since I am also teaching atm, I am super drawn to everything else that you’re talking about as well.
I was curious about the criticism that they gave you:
“a) my weak review of the literature, b) the sense that I wasn’t really taking myself anywhere new with the study, and c) a claim about equity that had me reaching beyond my data.”
Did they recommend ways of reducing or mitigating these concerns? Or rather, did you think about options of improving your research? If so, how did you approach that?
Dan Meyer
September 13, 2012 - 10:16 am -Jim P:
The solutions were a) write a better review of the literature, b) nothing, c) remove the claim about equity or do it more justice.
I didn’t do any of those things and no one expected me to either. I made lots of revisions in response to feedback throughout the process but those three bullet points were meant to inform future research, not a revision of my qualifier.
David Wees
September 13, 2012 - 11:24 am -Dan, I really want to get my PHD for the same reasons you have cited. I feel like I can make more of a difference, and get funding to do the kind of research I’m really interested in.
One problem: I earn the money for my family, that’s my role. I can’t for the life of me figure out how to continue doing that while I’m working on my PHD.
People have told me there is “some funding” for PHD programs but finding out exactly how much has been a bit challenging. It seems to vary somewhat depending on your success getting scholarships and grants, but no one claims they can support a family on their PHD grant money. So I’m stuck, unable to follow this path because of money.
Any suggestions?
Bruce Rhodewalt
September 13, 2012 - 1:43 pm -This site continues to inspire me. I completely ripped off all your ideas and presented them to my classes like this:
http://teachfluxion.wordpress.com/2012/09/07/cost-of-paint-priceless/
One of the most enjoyable lessons I’ve participated in. Thanks.
John Scammell
September 13, 2012 - 3:51 pm -Considering my current job requires me to produce performance assessment tasks, I am very interested in hearing what you learn working with the Shell Center.
@blaisej
September 13, 2012 - 10:45 pm -Thanks for the reading list. The first article I read was excellent, especially the first half: Egan, K. (1999). Education’s three old ideas, and a better idea. The author makes a compelling argument against three conflicting, old forces in education. It is very applicable to today’s educational environment. I’m not sure about his “new idea” but it may have been too cerebral for me.
I look forward to reading the others.
Dan Meyer
September 14, 2012 - 5:36 am -@David, really tough question there. I can only relate that Stanford, and some other institutions, cover tuition for four years –Â no questions asked. You have to work 20 hours a week (10 hours on two jobs each –Â either as a teaching assistant or a research assistant) but you don’t pay for school. Moreover, they pay you about $1,000 per month. It isn’t much but I’ve also managed to keep up some consulting on the side as a supplement.
One way or the other –Â either in lost income or in lost opportunity –Â grad school is expensive. Me, I find the expense worthwhile.
Andrew O'Grady
September 15, 2012 - 9:03 am -Good luck with your PhD. I did mine in engineering prior to becoming a teacher. It certainly is a challenge, but from what I saw, students who had already held a real job were able to appreciate it more than those who went right from undergrad (like myself).