Keith Devlin (@profkeithdevlin) invited Karim Ani (@mathalicious) and me to co-host a panel at his summit with Finland and American academics on our experience with social media in teaching. I captured it on my Flip camera and posted the video below.
I’ve also transcribed two passages I wanted on the record. One is my introductory remarks (starting at 9:30) which cover how I’ve used blogging to grow as an educator – from 2006 to present day. The other is some real talk about grad school (starting at 45:36) prefaced by, “I’ve never shared this with anybody here, least of all my adviser, who’s in attendance.”
Keith came upon me kind of halfway through my story of social media and teaching. The start of that story was when I didn’t have professional development options. Like I started in a big district with lots of them, big professional development days every month, huge festivals of teacher learning, and then I went to this school in the foothills of Santa Cruz where there was a total of six or eight math teachers in the entire district. There weren’t a lot of options. The people I worked with – they were much more senior than me. They had already pushed past or resolved or were not interested in the issues that I was interested in and needed to learn about. So for the cost of a few cups of coffee, I went online, made a website, got a blog software running and just started writing a journal and what happened for me as a young teacher was – I’m just grateful for it. That’s my primary stance here is one of gratitude because this could have gone any number of ways.
I wrote about everything I did. I’m not a bad writer so it wasn’t boring to read but people started coming around, people who were much more veteran than me, who had the same interests that I did, and essentially I found myself creating my own faculty lounge. And these people would just pepper me with criticism and flatter me with some compliments now and then if they liked it and I learned to – I figured out who was good to pay attention to, who the cranks were, that’s a key skill I had to develop. And, honestly, I have no empirical basis for this but I think I developed two years as a teacher for every one year I was in the classroom on account of all this criticism sanding off my rough edges and pivoting me away from some dead ends. You know I had my worksheet phase where I would obsess over font choices on worksheets or whatever and I had enough feedback that like “There is a limited shelf life for this.” And that was great.
So pursuing this thread of return on investment – 200% return on one year of teaching is how I saw it – and there’s plenty of other ways I saw that too where I would post curricula that I had spent a lot of time on. The first big one for me was a video series called Graphing Stories where kids would watch videos and then graph elements of it to introduce two-dimensional graphing. Took me like eighteen hours over a weekend to create the thing. And it was good and productive for my class. But I posted it online afterwards and just offered up the whole thing in an easy download and 6,000 people downloaded it in two weeks. And the process gave me lots of feedback which made it better for my students. Social media for me creates this very virtuous cycle where I become more productive, get better feedback, my kids win, other peoples’ kids win. And this narrative plays out in reverse with other people who blog and me. I am that person to them that they are to me. It’s something that I’m still trying to figure out. I posted all of my Algebra 1 curricula I developed. All my Geometry curricula is just there for easy download. This is kind of the lifestyle now.
I applied to give a talk for my regional math conference. You know, we have this small North California thing. Not like an NCTM plenary. It was like a twenty person thing in a classroom at a middle school but you know I put a small camera on a shelf and filmed it and put it online and just sent a note out and, you know, there’s twenty people in that room and I just checked it before I got here and 3,000 people have watched it in total. That’s passive. I didn’t do anything to pursue any of them. I just put it somewhere where it’s easily found and my message, the things I’m enthusiastic about, that I want to advocate, those get out there. And people find it. And they learn from me. And they critique me. And it’s all very, very positive.
People from news organizations will contact me and ask for my opinion on this or that. I’ve seen this happen to other bloggers who have their hobby horses and they pursue them relentlessly and I see them in every single article or TV press piece on, say, Khan Academy that there is. They’re always there. And you think about some teacher in some rural area who has no impact outside of 150 kids and six teachers in their department. They can have that worldwide, national impact with social media. So see it as an advocacy thing. See it as a professional development thing. Those are my two largest frames I have on it.
Real Talk About Grad School
Can I please follow up on that [Devlin’s comment about webcasts] and ask a question out here, of the academy? I’m a grad student in my second year and I’ve never shared this with anybody here, least of all my adviser, who’s in attendance, but I don’t understand the incentive structure for what you do and what I may do someday. You write amazing things and you study amazing things and you write them compellingly in journals that are not read by practitioners very often. They affect a lot of policy, which I think is a really good, top-down approach. But then I’m over here and I can post something that’s seen by 10,000 people overnight. That’s the number of subscribers I have to my blog right now. Or any number of these things. So the incentive seems strange to me. Like I don’t understand this brass ring I’m chasing. It seems like a strange prize at the end of a finish line of grad school. So there’s the question and then there’s also the encouragement. You have so many soapboxes available to you. Find a kid like me and ask him how to do a webcast or something. You have so many – and to restrict yourself to peer review, I don’t know. There’s very little upside to me, it seems.
[to my adviser] We should talk.
2012 Jan 22. I’m recalling Prof. Loeb:
By the time you see a paper in a journal, the field has moved on. (Loeb, 2010, lecture).
2012 Jan 22. Scott Elias started a series exploring the value and process of an education doctorate.
2012 Jan 25. Had that meeting with my adviser. I wasn’t sure how it’d play out since I kind of took an incredulous stance towards the academy in that panel. The upshot of the conversation is that she wants some help setting up a blog and she thinks I should look at some of this online teacher professional development we run here as a possibility for dissertation research. So … positive, I’d say.
For me the PhD wasn’t the brass ring. The brass ring was discovering something about (in my case) the physics of the ocean from a point of view that no one else ever had taken, and convincing a bunch of people who had studied that system really closely that what I was seeing made sense and wasn’t missing anything important (as far as we could all tell at the time). Or, as Ben puts it, “Having learning as a full-time job is really, really delicious.
From my vantage point, the brass ring of academia is not getting the message one already has to the most people but the search for the right message. Can they be done simultaneously? I don’t know. But, the fact that someone may have 10,000 followers does not mean that person possesses the truth. Many people read USA Today, but it’s still a crappy paper.
If we could really put much stock in this research, people would look at the results and agree that XYZ works and everybody would do it, and just leave the fiddling alone, but nobody has figured out what XYZ is, other than to make sure that kids come from higher income households and that their parents care about their education.
David WeesJanuary 19, 2012 - 8:48 am -
I’m looking toward a PHD program myself, really for two reasons. I want support in learning how to do high quality research from a mentor, and I’d like the support I get to be as personalized as I can get. It seems to me that 1 to 1 relationships are better for that.
As for the learning in the courses I will be required to take, some of that will be stuff that I know, and some of it will be stuff that people think is necessary to know, and I may not know what is “necessary” to know. I’m essentially paying for less restricted access to the body of knowledge considered important in my field, and curation of that body of knowledge.
As for the reward at the end, I’m agreement with you. It’s a hoop that we’ve developed as a society, and I can imagine that someone could develop in such a way that their peer review process is more organically through their network. Of course, most people do not have 10,000 subscribers.
I was considering (and am still considering) doing an ‘unPHD.’ It involves all of the work of a PHD, but the peer review process is a little bit less formal, it costs a lot less, and a position in a university getting to teach teachers is a lot less certain. See Uncollege: http://www.uncollege.org/
DarrenJanuary 19, 2012 - 9:34 am -
From what I understand, it is easier to get a dinner reservation if you call yourself “Dr. Meyer” over the phone.
John ScammellJanuary 19, 2012 - 10:07 am -
How’d the talk with your supervisor go?
Christie RansomJanuary 19, 2012 - 10:29 am -
All of this is fantastic, Dan, and so inspiring. I have been teaching about the same amount of time as you in a school in England and on Monday I led a cpd for my department on your talk. I have tried something similar in most lessons this week and I feel like I am refreshed and renewed to keep teaching outstanding lessons. Maths teaching can become very stale very quickly and I love the ideas you present about using social media and real world situations to bring timeless (and beautiful) mathematics into the present day. Keen to soak up as many new ideas as poss, so have been thinking a blog might be the way forward and this has affirmed the idea! Thank you.
Jason DyerJanuary 19, 2012 - 5:33 pm -
I can answer this a bit, working with various people at the University of Arizona last school year–
Most of them don’t have time. It is seriously insane what they have to do to get tenure. I will abstain from stories for privacy reasons, but you can find similar stories bubbling about the Internet.
Having said that, the chair of the math department has a blog:
Tools for the Common Core
Keith DevlinJanuary 19, 2012 - 7:31 pm -
Jason, after you get tenure, the demands on your time increase dramatically! They really do. I’m not complaining. It’s great work and I love it. But it consumes 12 to 16 hours of every weekday and 8-12 hours of each weekend day.
Jason DyerJanuary 19, 2012 - 9:08 pm -
@Keith: Wrr, I realize I phrased that wrong. But still, same point applies. It would help if blogs held more prestige in hiring/grants/etc.
LizzyJanuary 19, 2012 - 10:13 pm -
Thank you for putting up this post. I am one of those rural teachers, and I do my best for my 60 odd students, their parents and I pour my soul into my school community but yet I feel terribly lonely and isolated because I have so few other math educators to communicate with. Reading blogs like yours and has kept me going. I have used a lot of the lesson plan ideas teacher’s have presented on their blogs and I hope to share and get feedback on my own. Reading these blogs has been of so much more worth than going to conferences because I have the leisure to find answers to exactly the question I had and to I get to see how other teachers grapple with the nuts and bolts of teaching on a day to day basis.
Michael Paul GoldenbergJanuary 20, 2012 - 6:56 am -
I hesitate to comment, lest someone accuse me of sour grapes, as I have two uncompleted Ph.D programs (English at U of Florida and mathematics education at U of Michigan) on my permanent record (yes, THAT permanent record). But my sense is that in the foreseeable future, having a Ph.D may not be the obvious way to go if you want to make a difference. The problem is more in the minds of the beholders than in the reality of who matters, whose ideas gain traction, and how wide a swath those ideas are likely to cut.
It’s hard to think of a Ph.D these days whose ideas have as much influence and reach as many people as have those of Steve Jobs (I’d say “Bill Gates,” too, except that I’m not sure he’s ever had an original idea). Does Sal Khan have a doctorate? If so, it’s not in the area where he’s having so much influence (whether for good or ill I won’t comment here).
When someone weighs in on mathematics education, I don’t immediately accept or reject the ideas based on what degree(s) that person holds (or doesn’t). I tend to be a bit more skeptical if someone without a Ph.D in mathematics (or an impressive track record of peer reviewed publications) claims to have found a proof of the Riemann Hypothesis (or an elementary proof of FLT, or a refutation of Wiles’ proof of FLT), but that’s because I’m not able to evaluate advanced mathematics myself, for the most part. But if someone claims to have a fantastic approach to teaching mathematics in K-16, I’m confident I can read what she has to say with an effective critical eye.
Maybe Dan Meyer would have more impact quitting the Stanford doctoral program and finding a way that he can do some (limited) K-12 mathematics teaching (always good to have one’s own classroom as a lab) and invest the rest of his time and energy in “other projects” on the web or off. If I were in a position to hire and could get Dan Meyer to teach two classes a day at my high school . . . well, I’d jump at the chance and do whatever I could to work things out so that he could make a go of it in terms of the “other projects.”
And if I were funding research and innovation in mathematics education and Dan Meyer had an intriguing idea for an “other project” but needed/wanted to teach two classes a day at a high school, I’d jump at the chance to fund the project and do whatever I could to work things out so that he could make a go of it in terms of that classroom teaching time.
But will there be people out there actually in such positions who see things as I do, not only regarding Dan Meyer but other innovative educators? I don’t know, but I think it’s not an absurd notion. Probably why I’m wearing a cardboard belt.
AndyJanuary 20, 2012 - 8:16 am -
The point of an PhD is normally to allow one into the club — the club of professorship. That’s it. If you want to join the club and write peer-reviewed papers that are published in obscenely expensive journals that nobody reads, get a PhD.
If people are getting PhD’s so that they can make a policy impact, then they have been misled.
Why, other than getting a gold star in your profession (or to learn something, gasp) would anyone get a masters? For many fields, a masters means that you washed out of a PhD program.
We’re an odd society.
ElizabethJanuary 20, 2012 - 9:28 am -
Very thought-provoking post, Dan.
I got my credential and Master’s in Ed at Berkeley, and we took all of our Master’s level classes with the PhD students. At that time, my goal was to teach for 3—5 years and then get a PhD. I graduated with serious doubts about whether university research has much impact or applicability in schools. One of the PhD students expressed his misgivings that the program was really just preparing him to recreate a similar research program in another university, and I thought it was a valid concern.
I would guess that your path through the PhD is already substantially different from most students. Are other students in your program leading PD workshops around the country? Probably not. So you’re carving out a unique path and you’ll probably end up with a unique position at the end. That’s really cool. And, I’m less cynical/skeptical about the value of research these days. There are researchers fighting the good fight, and teachers fighting the good fight. We need folks who have credibility in both circles to keep the researchers grounded and give the teachers’ voices more power.
Belinda ThompsonJanuary 20, 2012 - 11:58 am -
As a former classroom teacher now in a Ph.D. program I am so happy to hear Dan pose the issue of the incentive structure in research at the university level. I’ve voiced this myself in several of my courses. Part of it may be that as a soft science the study of education has to assert that it is a true science by adhering to the other structures in place in academics. What if the journal article had to include the impact the work has had in K-12 or a description of how the work would be disseminated to those who could actually use it.
Over the course of my program I have often been frustrated by reading about a study that was conducted prior to my teaching stint that discredited a practice (or manipulative or technology or whatever) that I used. As a teacher, not only would I not have been the audience for the write-up of the study, but I would not have had access to the information in the first place. I think access is an even more critical issue for teachers than not being the audience of the work of educational researchers.
Belinda ThompsonJanuary 20, 2012 - 12:05 pm -
Oh, also wanted to comment on Michael Paul Goldenberg’s comment about teaching part-time. I would love to have a classroom of students to try out all the things I think I’ve learned, but the structure of public schools and the structure of grad school don’t necessarily mesh so well as to allow that. This is more evidence of some silo action in the world of education.
A final comment about isolation. It’s even possible to feel isolation in a large school or a large district when the school schedule precludes teacher collaboration during the regular workday. Social media is a fantastic way to address that, and I think the participation shows that teachers and researchers can have productive relationships and enlightening exchanges if given the opportunity.
dan goldnerJanuary 20, 2012 - 3:55 pm -
For me the PhD wasn’t the brass ring. The brass ring was discovering something about (in my case) the physics of the ocean from a point of view that no one else ever had taken, and convincing a bunch of people who had studied that system really closely that what I was seeing made sense and wasn’t missing anything important (as far as we could all tell at the time). Or, as Ben puts it, “Having learning as a full-time job is really, really delicious.”
What you didn’t say here is how many math teachers have joined the blogging community and now enjoy the benefits you describe because at some point in the past they googled “math teaching blog”, got lucky at blog.mrmeyer.com, and found amazing colleagues. Thanks, Dan.
dan goldnerJanuary 20, 2012 - 3:56 pm -
Sorry, I must have hashed the link. Ben’s quote is at http://researchinpractice.wordpress.com/2011/09/16/never-be-wobbly/
k morrow-leongJanuary 20, 2012 - 11:59 pm -
I too am in my 2nd year of a PhD, and I mostly appreciate the opportunity to learn from professors who have read more than I. But I also realize that reading and knowing more isn’t a productive contribution to student learning per se.
One of my advisers organizes summer professional development for inservice teachers and the group’s research goal is determining the impact this professional development has on instruction and learning. This I find professionally meaningful, but unless we conduct research and publish, it does not resonate in the academic community.
But, on the subject of access, I’m interested in the latest announcement from Apple, the ibook on the ipad. I am interested to know if anyone knows anything about it yet.
mr bombasticJanuary 21, 2012 - 7:30 am -
At one point Devlin raised the issue that the most knowledgeable folks with the most to offer do not have a presence on the web. I think that is exactly right. I want to hear from that small number of truly outstanding teachers that are clearly head and shoulders above all of us on our best day. I have picked up quite a bit from reading this blog. I think Kate Nowak has some very useful ideas as well. But, the person I would really like to have a blog is Dan or Kate, who, with 20 more years of teaching experience have developed into truly outstanding teachers.
I would also like to be able to watch videos of truly outstanding teachers. How is it possible that the schools of education can’t show their students what a week’s worth of lessons look like in an outstanding teacher’s classroom? It is like asking someone to learn how to play golf without letting them watch someone swing a club. I would like to see an education “researcher” make themselves useful by grabbing a video camera, a pile of consent forms, and hit the road looking for good classes to tape.
I don’t see the point of you chasing the brass ring either. Why are you chasing it?
James C.January 21, 2012 - 9:27 am -
I just want to reply to your comment on the video of outstanding teachers in action. My Assessment class used a textbook that contained DVD clips of teachers implementing assessment techniques in their classes. The videos were high quality, and I thought they were very useful to watch. Maybe not as much for an experienced teacher though, certainly not worth the cost of the textbook.
James C.January 21, 2012 - 9:33 am -
Typically in most industries you see progress rapidly in the private sector, a little slower in academia, and slowest in public policy.
What we’re seeing here is kind of the same thing. The fastest moving ideas are the people on the front lines (i.e. the blogging teachers). Academic circles move slower, and then the institutions, the schools and boards of education move slowest.
Dan MeyerJanuary 22, 2012 - 6:13 pm -
I’m not. I may submit research to peer-reviewed journals but that’s a long way down the list of goals for my time at Stanford.
Janet | expateducator.comJanuary 23, 2012 - 4:46 am -
If I could advise grad school programs…
– Model differentiation. Not all of us need the same lessons (nor do our students).
– Require blogs – at least one blog post reflection for every formal paper. Use posts as formative assessments.
– Require growth of PLN. Others can help make grad school assignments more meaningful.
– One peer-reviewed piece is okay. We might as well know what it feels like to have our work picked apart.
Nate LJanuary 23, 2012 - 8:35 pm -
Hey Dan, I’ve been interested myself in getting possibly returning for a PHD in Ed…
This is an interesting topic
I’m also wondering your thoughts about Apple’s new anouncement about Ipads & Ibooks2 Textbooks.
Even more so, the Author software and some of your ideas from your Act1, 2,3 stuff.
Here’s my blog: trying to get going, would love for you to check it out:
ClimeguyJanuary 25, 2012 - 7:09 am -
Nice act of courage, Dan. I’m sorry your advisor didn’t respond during the filming of the event. It brings back memories for me. I opted out of a EdD (early 80s) program at Teachers College which as a first course required that I write a scripted “mini” dissertation. I assume the dept. chair wanted to sort out any potential drop outs of the program immediately by giving them something “hard” to do right off the bat. While struggling with it, I decided I would much rather work on a book that I was dreaming of writing that would help me in my consulting endeavors. (I finished it in 1988 and got some interesting consulting gigs as a result.) Years later, I did feel somewhat guilty about not getting that degree, but realized during my years with Stevens/CIESE that I was having too much fun doing what I was doing now (circa mid 90s) and it no longer made sense to jump through hoops just to get more credentialled to do the work I was already doing well. Looking back on it now, I wouldn’t have done it any differently. A close friend who did complete the program at TC at the time told me that it wasn’t about the degree for her but the wonderful life of being a graduate student. (BTW my friend started the first Math Ed microcomputer lab at TC in 1980. Those were heady times.) She was right. Being her colleague at the time while I was teaching in private school in NYC and P/T getting my Masters was the most formative time in my career.
Cameron ByerleyJanuary 25, 2012 - 8:44 am -
I think about the PhD/impact issue all of the time. I see the absolutely brilliant things my PhD adviser writes about student thinking and learning in math education and I wonder if I can use social media to translate Pat Thompson to the world.
I’m personally glad I’m in a PhD program because I’ve drastically improved my understanding of the issues in math education. It’s getting to the point where I can go to a conference and analyze how the lack of focus on student thinking in a project is foiling their efforts to get the results they are seeking in student achievement.
But the part I never tell to my adviser, is that perhaps I want to take all the knowledge he’s given me and start a website like Mathalicious or Khan, except actually use research on student thinking and learning to make the products better. It frustrates me when I see major issues in Khan’s videos(like reducing proportional thinking to cross multiplication and skipping quantitative reasoning) and know that Gates is spending a lot to translate them to other languages. It’s perpetuating the same old issues our culture has always had(see the Teaching Gap for details). Of course I need to have the right mix of creativity and personality to sell these ideas to the masses-maybe that is where my stint as drama club president comes in. And I’m inspired by what you are doing as a speaker as I know we have about the same skills sets in terms of understanding of math teaching-I almost went to Stanford and would have been in your classes. But you’ve got the social media piece down.
I’m sure I’ll keep brainstorming alternative careers in math education as one of your readers mentioned.
KarimJanuary 25, 2012 - 8:00 pm -
Cameron’s got game. She’s a sharp cookie, which makes no sense culinarily, but is true nonetheless.
Cameron ByerleyJanuary 25, 2012 - 9:06 pm -
That means a lot from someone who is getting famous in math circles-I’m so glad you are dedicating your time and creativity to math education :) I so enjoyed getting to talk fractions with you.
You have some lesson writing talent that no amount of education is going to make up for!
AlexJanuary 26, 2012 - 8:21 am -
An immediate disclaimer: I am married to a tenure track professor at a school of ed (not in math). I see the peer review process as a search for truth. That is, a way of finding what we can say we truly know. I know that it has sharpened my wife’s thinking about teaching and learning in her discipline.
Blogs seem (to me) to be more about disseminating what the author already believe to be true. I think that many people believe that they know what good teaching is without any real proof. Why else would people latch onto every fad in education? I fully admit that what I do in the classroom is mostly a product of what I believe good teaching to be and not what I know good teaching to be.
From my vantage point, the brass ring of academia is not getting the message one already has to the most people but the search for the right message. Can they be done simultaneously? I don’t know. But, the fact that someone may have 10,000 followers does not mean that person possess the truth. Many people read USA Today, but it’s still a crappy paper.
Dan MeyerJanuary 26, 2012 - 10:56 am -
I pulled comments from Dan Goldner and Alex up to the main floor. Both nicely describe the appeal of scholarship and the academy. Here’s Alex:
Pushing back a bit: once you have the right message and your goal is to get it to as many people as possible, I don’t see the upside of peer-reviewed journals. That’s where ideas go to not be read by practitioners anywhere. That was my point on the panel. Not that readership correlates to quality.
Also, for anyone who’s interested, the conversation with my adviser about my remarks on the panel was productive. I wrote a few sentences about it above.
k morrow-leongJanuary 26, 2012 - 11:16 am -
I’ve been thinking that each peer-reviewed journal article published should be required to contain a version directed at the practitioner, if only just out of respect for their position. If you can’t describe it in a way that makes sense to the practicing teachers, your paper should set enough groundwork that they can at least see why the idea is important.
This means that JRME, PME, JMTE, etc. should each have a companion volume, published in iBook or otherwise online, that offers this knowledge to those who might use it directly.
JRMEp, PMEp, and JMTEp, etc. …
Dan MeyerJanuary 26, 2012 - 11:24 am -
@Kim, I like the idea. It seems that kind of “translation” is up to the author, though, not the journal editor. Those aren’t trivial re-writes either. My adviser publishes in journals and in mass-market paperback. She gives talks at PME but also at NCTM and CMC and for local audiences. I admire the skill it takes to translate the same message for different audiences.
mr bombasticJanuary 26, 2012 - 1:20 pm -
Is there really anything of interest to teachers in peer reviewed education journals? The notion that teachers are being deprived of valuable education research hidden away in academic journals seems far fetched to say the least.
Also, I suspect that a not so small role for the education journals is establishing and maintaining a pecking order in a discipline that is dominated by opinion and anecdotal evidence.
AlexJanuary 26, 2012 - 1:54 pm -
From @Dan: Pushing back a bit: once you have the right message and your goal is to get it to as many people as possible, I don’t see the upside of peer-reviewed journals.
Agreed. The relatively small number of researchers I know struggle with that. But, the researchers I know who focus only on spreading ideas don’t really seem to be researchers. It’s almost as if research and spreading the word are both more-than-full-time jobs and so are difficult to do simultaneously. That’s not to say it can’t be done, but sacrificing one for the other (no matter which is the other) has pitfalls.
From @bombastic: Is there really anything of interest to teachers in peer reviewed education journals?
We agree that there is a lot of low quality ed research, but to say there is nothing of value seems too extreme, to me. If academic research is anecdotal then what teachers think of as good teaching is even worse. When was the last time you heard a teacher say that something worked in his/her classroom and had real evidence to back it up? How many of us say about a colleague, “Teacher x is a god/bad teacher,” without any real knowledge of what kids in that room learn? I believe we benefit from an exploration of teaching and learning that is based on more than gut feel.
AndyJanuary 26, 2012 - 2:20 pm -
But that is the problem – determining causation in really hard – even getting strong correlations is difficult and most studies are so small that the effect needs to be huge to be really significant. Even in the studies of individual teachers, where you would think impact correlations would be high, they aren’t.
The trend here these days is to worry about engagement and student-centered learning, but studies from around the world show that lots of different teaching styles can lead to good results and bad results – which suggests that the “style” of teaching may not really be critical.
The other problem is that there are lots of people going to school to get PhD’s in ed, and many of them would be out of business if you were really honest about impact — that this method or that curriculum really didn’t make that much difference. There is just too much bias with researchers, cooperating schools, teachers and so on to get good, clean results.
If we could really put much stock in this research, people would look at the results and agree that XYZ works and everybody would do it, and just leave the fiddling alone, but nobody has figured out what XYZ is, other than to make sure that kids come from higher income households and that their parent’s care about their education.
Belinda ThompsonJanuary 26, 2012 - 2:41 pm -
Totally digging the thought provoking comments here! Seems as if several of us in the throes of graduate school are contemplating what happens after we walk across the stage. If we’re seeking opportunities to gain and share knowledge now, my guess is that this will continue. So, where will we land? What system will support , encourage, or even require these endeavors?
Can we also think about this from both directions? If promotion or membership in academia is tied to impact on some segment of the education system, couldn’t some onus be placed on K-12 to seek out or participate in research activities? I know there are universities and K-12 schools who work closely, but there’s no system (that I know of) that facilitates such relationships in general.
My area of interest is pretty specific: teaching and learning fraction concepts in the middle grades. Fractions are hard to teach and hard to learn. I’ll likely work on it for my entire career. However, the process of recruiting districts and teachers to participate in studies is time-consuming, much less the logistics of carrying out a study in a live school setting. There just simply isn’t a culture of systematic evidence gathering and analysis in our K-12 education system, and it’s my ongoing responsibility to convince administrators and teachers that participation in research projects is valuable to the system at large.
CameronJanuary 26, 2012 - 5:03 pm -
Just reading Izack’s 2008 paper on MKT for teaching fractions. Such a helpful lit review section!
I like your ideas about making it easier for me to collect research :)
I think determining causation in education is really hard. The system is so complex. I think that Teaching Experiments(Steffe and Thompson) are an example of research that can help teachers by giving thoughtful examples of working with kids that don’t rely on proving causation. An example of this working is Cognitively Guided Instruction where teachers were taught what researchers knew about student thinking in elementary school and how to listen for it and use it in instruction.
For info see
Doing qualitative work well requires some serious thought about establishing evidence and making claims, but I think there can be good qualitative work even if lots of it is of poor quality. I actually do think that there are huge insights into math learning in the research that most teachers have never come into contact with, even if they have gone through teacher ed programs. Certainly my departments never even got close to talking about what research could say to explain problems and most of our solutions and explanations of problems were, in hindsight, way off the mark.
mr bombasticJanuary 26, 2012 - 5:05 pm -
@Alex, Agreed a small percentage of academic research is good, and a small percentage of the good research is relevant to those in the trenches. My point is that if all these folks had dan post their research on his blog, there still wouldn’t be a lot of folks reading it.
Using anecdotal evidence from my own classroom to adjust my own lessons is quite a bit different from a researcher using anecdotal evidence from other teachers classrooms to suggest ways for me to improve my classroom. Not saying there isn’t value in this – but the researchers often seem to be heavily biased in their observations.
I think gut feeling is actually better than most of the data driven non-sense. Somehow common sense goes out the window for some people when they start looking at numbers. The attempts to quantify differences in eduation research are nearly always dubious at best. It is just too complicated and too expensive to do meaningful studies. More often than not, the few valid studies that are done show no significant differences (i.e. The US dept of education’s Nothing Works Clearinghouse).
k morrow-leongJanuary 26, 2012 - 5:54 pm -
The onus is on academia to make their research relevant to practitioners. If there were no students or teachers, there would be no reason for education researchers to exist. Research assistants, freshly out of the classroom or still teaching may be the best translators. But since there is currently no need, the skill is not cultivated.
I agree with Dan: it is a skill to translate into layspeak. But we should also expect it. (And I have read your adviser’s scholarly and lay work, and agree that it is skillful.)
SeanJanuary 26, 2012 - 10:17 pm -
“But that is the problem — determining causation in really hard — even getting strong correlations is difficult and most studies are so small that the effect needs to be huge to be really significant…If we could really put much stock in this research, people would look at the results and agree that XYZ works and everybody would do it, and just leave the fiddling alone, but nobody has figured out what XYZ is, other than to make sure that kids come from higher income households and that their parent’s care about their education.”
This position lacks nuance. I’m interested in your thoughts on the following reports, each of which have huge effect sizes and are highly prescriptive in the XYZ sense.
“In the 2010-2011 school year, we implemented five strategies gleaned from practices in successful charter schools — increased instructional time, a more rigorous approach to building human capital, high-dosage tutoring, frequent use of data to inform instruction, and a culture of high expectations — in nine of the lowest performing schools in Houston, Texas. We show that the average impact of these changes on student achievement is 0.277 standard deviations in math and 0.062 standard deviations in reading.”
Angrist et. al (2011):
“Our analysis focuses on special needs students that may be underserved. The results show average achievement gains of 0.36 standard deviations in math and 0.12 standard deviations in reading for each year spent at KIPP Lynn, with the largest gains coming from the LEP, SPED, and low-achievement groups. The average reading gains are driven almost completely by SPED and LEP students, whose reading scores rise by roughly 0.35 standard deviations for each year spent at KIPP Lynn.”
Kane & Staiger, (2012):
“We were able to identify larger differences using the three-measure combination of value-added, student feedback, and observations. By combining these, the difference in student achievement gains in another school year for those identified in the top and bottom quartiles was 0.16 to 0.21 standard deviations. The largest of these is nearly equivalent to the effect of a full year of schooling (0.25 standard deviations).”
AndyJanuary 27, 2012 - 6:30 am -
It would be really helpful if you would provide some references.
SeanJanuary 27, 2012 - 6:43 am -
3. Kane & Staiger
AndyJanuary 27, 2012 - 8:47 am -
I’m super-familiar with CGI and it is a good example. Put the studies aside for a minute. CGI has been around and in use for a while now. One would think that if IN PRACTICE it really made a difference that 6th grade math scores for schools that used this method would pop. This would be obvious and touted by CGI proponents for all to drool over. Has this happened? Not that I am aware of.
What conclusions would you draw? Should one persist, saying that the research says that it’s great, or would you perhaps say that the research was of limited value in practice, or even flawed. Our local school which has been spending every free moment of professional development on CGI for years has seen 6th grade math scores fall, though not significantly (but still enough to be put on the NCLB watch list). So, either the kids are just getting stupider (could be :) or CGI doesn’t have the impact that the research leads one to believe. One school doesn’t make a great sample, but over the years it’s still probably way more kids than have been in any CGI research project. But again, let’s see the real-life numbers.
P.S. – We have 4th graders still drawing 15 caterpillars with 6 legs each and counting to figure out 15 x 6. Sure, we’d like to “lead” them to other, more efficient methods, but it doesn’t always work that way. Our school has even dropped the computation section of the standardized tests and middle school teachers are complaining that kids are coming in less prepared than ever. There are many teachers who think CGI is the greatest thing since sliced bread – but if you are going to measure results by standardized tests, in the end it hasn’t done much for the kids here :(
Scott McLeodJanuary 27, 2012 - 11:33 am -
“Having learning as a full-time job is really, really delicious”
I’ll echo that. You’re an insatiable learner and questioner, Dan. If you can get someone to pay you for that, it’s awesome.
And, yes, you’re also correct that we in academe need to utilize other methods beyond “peer-reviewed journals that educators don’t read” in order to get our knowledge and message out. If we’re invisible, we’re irrelevant.
Jason DyerJanuary 27, 2012 - 1:17 pm -
@Sean: I notice in the Angrist is this bit:
KIPP Lynn typically asked 5th grade applicants to repeat. These applicants might be expected
to do better on 5th grade MCAS tests just by virtue of repeating. We therefore assume that all 5th
grade applicants repeat and look only at their 6th grade and higher scores.
This strikes me as fairly significant to that group of students just past the 5th grade and makes me wonder what sorts of policies the schools has beyond the generic KIPP ones.
In any case plenty of schools both charter and public have had turnaround success before but duplicating their success is another matter; at the high school I teach at we modeled our recent reform efforts around a public school that had eye-popping numbers but we have not found anything close to the same success even with efforts that look identical on paper.
Even given all that I believe there are certain principles at a school policy level that are well-established through research, but the math-teaching level (the concern of Cognitive Guided Instruction and so forth) is another quandry altogether.
AndyJanuary 27, 2012 - 1:35 pm -
I don’t know that I’ll have the time to go through the papers completely, but I’ll note one item on the Kane paper. What happened in those schools, apparently, was real and great remediation in math in some grades — this is good, of course. But, the report is clear that the kids coming in to those schools were at a much lower level than the comparison schools or other district schools, making improvement perhaps easier?
But that aside, what can one really make of these results? There is no way to disaggregate impacts of the various changes made at the schools (new admin, mostly new teachers, tutoring, longer days, longer year). And most districts don’t have the resources to do what these schools did (over 2K/year in additional cost/student). The study isn’t long enough to tell us if the system can repeat at bringing gains to students who have been remediated. One also has to wonder — since the math outcome isn’t in any way correlated with the LA outcome — what is really going on.
I really don’t like the implication that throwing out the admin and many teachers was positive for the school, when the evidence – from a report standpoint – doesn’t support that. Still, the school may be a better place than the school in the past by other measures that we should care about, but really all we know is that they did a good job remediating math.
I’m also troubled that all of these studies are done by people who seem to have a bias – a vested interest in charters. It sure would be nice if people would throw their hypotheses to other, independent, researchers to provide a better basis for drawing conclusions.
CameronJanuary 27, 2012 - 1:49 pm -
My familiarity with CGI is only based on reading the research-I’m sure you have a much better understanding of how it works when scaled up. What I read involved careful work with teachers by researchers who had extensive backgrounds in math education.
If I was trying to figure out why it didn’t scale up I’d ask:
1. How many hours of professional development were offered?
2. Who conducted the PD and what was their expertise in student thinking?
3. Did teachers implement the program with fidelity or were their issues such as support, or teacher knowledge holding them back.
I do completely think that if CGI can’t be scaled up because it depends on having too many hours of PD with experts who don’t have much time then that does reflect poorly on the actual effectiveness of the program. Before deeming their ideas a failure, I’d have to wonder about how well they were implemented. Most districts don’t have elementary teachers with the knowledge to implement that type of thing, and given that issue,perhaps CGI wasn’t the solution.
Also-I once asked a ninth grader. What is 3 times 7? She told me in all seriousness “Miss, that one doesn’t exist.” I would have been way happier if she had drawn a caterpillar than basically telling me that multiplication facts only exist if she remembers the teacher saying them to her. I like that the caterpillar kids will at least know that there is always a meaning to multiplication and how to figure it out. I do know that multiplication schemes emerge from counting schemes(see Steffe’s work) but I don’t know if 4th graders should be able to generalize the meaning of multiplication or if it is appropriate for them to draw pictures of it. I do know that at my university we have pre-service secondary teachers draw pictures of multiplication and it can be really hard for them to do.
And yes, I do realize that in later math courses you have to know how to multiply quickly with large numbers.
SeanJanuary 27, 2012 - 2:15 pm -
“Even given all that I believe there are certain principles at a school policy level that are well-established through research, but the math-teaching level (the concern of Cognitive Guided Instruction and so forth) is another quandry altogether.”
Well-said. The instructional core (student-teacher-content) that Dan spoke to in his recent IBooks censure is really the holy grail. Are these high-performing schools changing that dynamic in some critical way that we can learn from? I strongly doubt that any of the schools in those studies revolutionized math pedagogy, but what *did* they do?
I guess the conclusions you draw from the studies is a referendum on how you view state achievement tests: either a) a sucker’s game or b) meaningful information on schools, teachers, and students. To the degree that you believe the latter: those math gains are enormous, and it wouldn’t be unreasonable to look at best practices around math instruction in those schools.
“But that aside, what can one really make of these results? There is no way to disaggregate impacts of the various changes made at the schools (new admin, mostly new teachers, tutoring, longer days, longer year). And most districts don’t have the resources to do what these schools did (over 2K/year in additional cost/student).”
I don’t understand that first point. Fryer’s concludes that it takes a transformed culture, not one individual component. Is there an need to disaggregate?
Agreed on the cost (although he does some beautiful work on the Social Return on Investment in one of the appendices). 2k a year extra isn’t cheap. I also think he understates the political challenges, which are far greater in most districts. He also got lucky with a bold Superintendent.
Raymond JohnsonJanuary 27, 2012 - 10:34 pm -
All week I’ve been looking forward to coming back to this post to read the comments, and I haven’t been disappointed. I’m still very much a math educator at heart, but this conversation tugs hard at my two other loves, education policy and how we share and value information. As much as I want to focus purely on the teaching and learning of math, I can’t stop thinking about how badly we need to establish new norms for education research. A few things I keep thinking:
I believe teachers are smart, curious people, and I reject the notion that we researchers, the “knowledge elite,” should be deciding on their behalf that there’s knowledge to which they don’t need free and open access. Yes, some (most?) research might be things they find irrelevant or incomprehensible, but that should be for them to decide, and they deserve help from “translators” who speak both languages.
Do we need visionary thinkers who can see beyond the horizon and search for yet-unsought knowledge? Certainly. But I’m not sure with the current state of education, including negative attitudes toward schools of education, that researchers can afford the luxury of *not* listening to the questions teachers are asking. I think there are many researchers who would agree with this belief, but haven’t figured out how much the internet has made listening to teachers so much easier than it used to be.
I rarely talk to any researcher who doesn’t want to have an impact at the classroom level. Those same researchers express frustration at how the current system in academia restricts them from having that impact. But rare is a researcher who expresses the belief that they have the power to change the system, and it’s a cruel irony how intelligent scholars can spend a career discovering ways to improve educational systems other than the one they work in.
A fleeting thought: There’s a tendency for some math teachers to have a sadistic side: “Learning math was hard and uncomfortable for me, and so it shall be for you.” (I’ll admit it…I have been guilty of this, particularly in my first year of teaching.) Does the same thing happen to researchers? I think we preserve some bad traditions because somehow we fear our accomplishments will lose value if we discard the system that provided for them.
IainJanuary 28, 2012 - 4:05 am -
Academic Journals vs Social Media is just a sign of the times.
Like vinyl vs CD it will change into something else like MP3 vs CD then the Cloud vs MP3.
I very much think with the speed of advance of technology and the growth of connectivity that Academia must adapt or become like the vinyl in the attic.
I think we are kidding ourselves to believe that papers written in journals are being read by the teachers in the classroom. The time lag in new research being filtered out to those who are practitioners is way too long. While the researchers are keen to see things put into practice their ‘reach’ at present is very limited. The students may have graduated by the time the ‘latest stuff’ reaches the humble teachers who also want to make a difference to their students.
I am sure most involved involved wants to do better but it’s a matter of politics, cost and accepting a massive change to the way things have worked from the dawn of time in academia, that is required to do thing differently. I will not hold my breath.
Conversely you can read blogs like Dan’s try things out, see if it works, get involved in the discussion and create a new order that believes the world is not flat because we have not fallen off the edge when we have travelled beyond where others have said we cannot go.
k morrow-leongFebruary 27, 2012 - 7:59 pm -
It’s like they heard us…
“Reporting Research for Practitioners: Proposed Guidelines”
NCTM Research Committee
March 2012, Volume 43, Issue 2, Page 126
The NCTM Research Committee developed this article to address a distinctly important activity that links research and practice: writing research-based articles for practitioner journals. Six guiding principles are described.