2011 July 8: Labaree just sent our class this same sermonette only now edited for publication. So take your pick. Polished or off-the-cuff.
This is a transcript of the “sermonette” David Labaree (author of two of the best papers I cited last week) delivered last week at the end of Stanford’s spring quarter proseminar. He gave me permission to share it with you. The tl;dr version is this:
- Be irrelevant.
- Be wrong.
- Be lazy.
Hell of an enticement, right? I’m posting it primarily for myself. I’m positive I’ll revisit it and find it intriguing in different ways the more I work in this field.
June 1, 2011
Be irrelevant. Even in a field where the problems of the field are so important and so demanding, the argument that Augier and March (2007) make is really smart. They say, on the one hand, the push to be relevant causes two kinds of problems. One has to do with what they call myopia and the other is ambiguity.
The myopia problem is that you’re looking at something that’s presented to you. Fix this. And so you burrow into this. But this is typically located in time and space. And the circumstances are such that you pull it in close. (That’s why he calls it myopia. It’s a nice image. You pull something up close if you have myopia. You’re nearsighted.) And so in the process of looking at it up close, the whole context disappears. And you start treating the problem as though it were not connected to that context. And then you often end up engineering a change that may or may not work in that setting but is not transferable because it’s actually involving things that are contingent on a particular context – a particular time and a particular place – and efforts to then generalize from that show that the insight was actually pretty irrelevant in terms of workability.
Also in terms of time. If you’re burrowing in too much on fixing the particular problem, by the time the work gets out, that problem has already evolved into something else. That was then, this is now. And even in that setting it may not work. It has already been outgrown because you are missing the evolutionary component of what’s going on.
So there’s a certain sense in which relevant research may actually have a very short shelf life. It may start to smell bad after awhile. It may have to have a buy-by date on it that says “After this, it may not be good anymore. It may not apply in any other location.” So in some ways, not trying to be so relevant may actually come up with insights that are more transportable, more useful, and are actually more applicable, even though that wasn’t your intention.
The other issue they raise that makes me say “be irrelevant” is that relevance is kind of a rhetorical plane and one of the things you have to say is “relevant for whom?” Is it relevant in the schools? Is it relevant for teachers? For students? For administrators? For superintendents? For policymakers? For politicians? For parents? It depends. Maybe what’s relevant for one is not good for the other. High-stakes testing is highly relevant for policymakers in order to make the claim of accountability in schools. It may be very harmful for teachers and students. So it’s relevant. Research supports it but it may be a relevance that depends very much on a relevance “for whom?” That’s a claim that’s not generic. It has to be established but it’s not generalizable.
There’s also the relevance “for what?” For what end? What are we trying to accomplish in schooling? Are we trying to make better citizens or more productive workers or help people get ahead or reduce social inequality or what are we trying to do? Well, the relevance of the research depends on the relevance to which of these claims it’s focusing on. One argument is that in some ways it’s a fool’s game to try to be too relevant in a field like this and, counter-intuitively, the most useful research may be the stuff that doesn’t seem to have an immediate application when you’re actually doing it. And an effort to be slavishly useful may give your work a limited purview and a very short shelf life.
All right. One piece of advice that nobody’s going to follow: “Be irrelevant.” Another one nobody’s going to follow, and that’s “be wrong.”
I think one of the dangers in programs like ours is that we encourage people to find answers, to be right, and that makes you risk averse. And my argument is that it’s much more useful to be interesting, and to provoke thought with your ideas, even if you’re wrong, than it is to be right in a manner that’s not very interesting, not very provocative, and not very likely to spur anyone else to do anything.
You never establish claims for all time. Truth is an ideal you pursue but you never reach it. And if you ever waited to nail down everything before you published something, you would never publish anything. Whatever you do, you have to recognize it’s going to be a partial statement. It’s going to be at best a partial truth. It’s something that’s true under certain circumstances and under certain conditions and with certain limitations about it and that’s enough, actually.
You’re not in the position where you need to make everybody’s counterargument to your argument. You just need to make your argument effectively and say to yourself, “Is this something that’s not in the conversation that should be in the conversation? If so, I should get it out there. And I can picture what some people will say in response but I don’t have to make that. Let them make that. I want to make a strong case in this direction. I don’t want it to be easily dismissed or laughable. I want it to have solidity, validity, and rhetorical effectiveness but it’s not my position to find out what the absolute final truth is because that’s not findable in my lifetime.
“So I’m going to be part of a conversation and the conversation is what matters and I’m going to learn from the conversation and in the process I’m going to revise what I did and I’m going to admit that some parts of that were wrong and I’ll move ahead and I’ll publish something else that is also a contribution to the literature. It helps with the conversation, but it also has plenty of possible responses to it. I’m going to learn from those responses too and I’m going to continue working on this in a somewhat new form after having acknowledged that certain parts of what I did before were not that good.”
That’s okay. That’s actually considered a successful career. That’s doing your job as as a social scientist. If you’re trying to nail it down and be right you probably won’t publish anything. You’ll keep waiting until you get it right. It has to be good enough to be provocative. Research is a provocation of thought. If you’re provoking thought with people and if you’re giving them a slice through a situation that’s a little different, that makes them think and reframe their understanding of something in an interesting way, that’s a successful piece of research. Even if it’s wrong, in a lot of ways.
As you know, it’s very easy to take even the best study and trash it. On methodological, theoretical, or other grounds. So that’s not the test of a good study. The test of good studies is, did it have an impact on you? Did it provoke your thinking? If so, it’s worth doing.
Be irrelevant. Be wrong. Third one. Don’t tell your advisor about this, but “be lazy.”
There’s a real danger in educational research that you just plow ahead. “I’ve got a big pile of data in front of me. I’m just gonna wrestle with it. I’m gonna run this data set using every single test in all of the statistical programs I’ve got and keep plowing ahead until I’ve got every permutation. I’ve got all this qualitative data. I’ve got all these tapes. I’ve got interview tapes. All these other kinds of things. So what I need to do first, of course, is to transcribe everything and then to code everything. And then try to put it all together.”
No – don’t do that. Don’t do that. You want to transcribe very little of it. Most of it’s garbage. Most of it’s noise. You were there listening. You know what was in there. You don’t have to transcribe every bit. Same thing with a big data set. You shouldn’t be wallowing in the data hoping it’s going to speak to you. It won’t. You have to make it speak. You’re looking for the music in the data. Most of what’s in data is noise. So your task is not to somehow encompass the whole thing. It’s to find a strategic route through the data that provides some kind of insight that’s not out there in the literature right now.
And often that means not being diligent. Diligence can be a dangerous trait in a grad student. It means “I’m just plodding ahead day every day. I’m going through another test. I’m transcribing another tape. I’m doing research.” No. You’re not. You’re transcribing tapes. Research means you’re actually trying to figure something out, you’re thinking your way through it.
Shortcuts are very nice. Shortcuts. “Do I have to go through all these data? Maybe only some of it matters. Maybe some of that whole issue is over there.”
I spent two years of my life working with a quantitative data set that I generated, coded, keypunched, analyzed, and had print-outs coming out my ears and it ended up when I published the book, I had a colleague, David Cohen. He looked at the book and said, “All of the data you had in there seem to be a footnote to the claim, ‘Central High School had meritocratic achievement.’” Two years of my life. A footnote. It turns out it was an interesting finding. It was counter-intuitive. But the actual interesting stuff was elsewhere in the data that didn’t take me two years of my life plowing through all of the stuff.
So don’t ignore the low hanging fruit and don’t assume that the only way to get from here to there is the longest possible route through the most amazing morass of data. It’s okay to think your way through and around a problem. That’s a good thing to do. Sometimes you find something and you’re gonna have to plow through it. But you want to have some confidence that you’re doing it for a good cause and you’re not just doing it in a kind of Stephen Colbert way. It’s “researchiness.” Researchiness means “I need to analyze data. It’s what researchers do. Give me some data. I need some more data.”
No, you’re supposed to come up with something interesting to say and it may be that only a little piece of data are actually germane to that and it may be that it’s an entirely different data set way over there that you want to be working on so why waste your time on this.
So as I said file that way. Never follow any of this. Don’t tell your adviser about this. But you might want to keep it in the back of your mind as a kind of cautionary tale about how you don’t want to get caught up in the aphorisms and the common senses of what research is. You have to keep in mind, “What am I doing this for? What am I trying to do here? What am I trying to get out of this? And how can I go about doing that in a way that’s productive and not just busy?”
Augier, Mie & March, James G. (2007). The pursuit of relevance in management education. California Management Review, 49(3) (Spring), 129-146.
markJuly 2, 2011 - 8:03 am -
I liked the first too points but the last one, being lazy with your data? picking out the data you think is important or interesting? I think the risk of confirmation bias is huge.
DamianJuly 2, 2011 - 8:25 am -
Thanks for this food for thought, Dan. Tucking it away into Evernote as I begin my own doctoral program in education this fall.
MyleneJuly 2, 2011 - 9:17 am -
Interesting point about willingness to be wrong.
The Journal of Engineering Education published an article recently that outlined education research philosophies used in different parts of the world. The authors suggest that in the US, quality of research tends to be judged according to correct use of methods, while in northern Europe, for example, it is judged according to quality of insights generated.
blaw0013July 2, 2011 - 9:49 am -
“The smart way to keep people passive and obedient is to strictly limit the spectrum of acceptable opinion, but allow very lively debate within that spectrum – even encourage the more critical and dissident views. That gives people the sense that there’s free thinking going on, while all the time the presuppositions of the system are being reinforced by the limits put on the range of the debate.”
— Noam Chomsky (1998)
gasstationwithoutpumpsJuly 2, 2011 - 9:56 am -
As a scientist and engineer, Labaree’s view of the role of research is disturbing to me. It is not ok to be wrong in the presentation of your results–it is ok to be incomplete, but you *should* be thinking about the possible and probable artifacts of the research method. It is not the role of research to raise interesting, wrong ideas (we have thousands of people doing that already in education), but to provide solid support.
This rather cavalier approach to the fundamentals of research seems more appropriate for a propagandist or a politician than an academic.
blaw0013July 2, 2011 - 10:22 am -
I read the three points as sarcasm; critique of modern research practice. I read Dan’s commentary as an activist’s response to the crap that is produced in the name of social science (educational) research, that grand effort to describe to the blind person what is. He argues a more powerful purpose for research is to make people think.
I suggest this provocation pushes the researcher to consider his/her conceptual framework quite carefully, at the macro-theoretical level: what is your ontological position? What is the status of reality, objectivity, subjectivity, truth, relativism? What is your epistemological foundation? (a ridiculous few math educators have left behind behaviorism as a theory for learning) And on…
Visit the announcement for a gathering of math educators who do intentionally conduct research to provoke thinking at .
Kate E Farb-JohnsonJuly 5, 2011 - 5:54 am -
I see Labaree’s first point as an strong critique of the idea that educational research should start with a problem. This idea bothered me when I first studied educational research, because I think that scientific research starts with a question, something the researcher doesn’t know and wants to. Starting with a problem to solve leads to confirmation bias, as well as huge gaps in the knowledge base of educational research.
The second point could be a response to the limited nature of educational research methods. Two major factors make it hard have a lot of confidence in even the best educational research. One is that education is contextual: a teaching technique that is effective in one class with certain students in a given school might be a disaster in another context. Looking a broad range of contexts is often prohibitively time consuming, so results are often only tested in one context. The second problem is that it’s nearly impossible to run a controlled experiment in education. Partly this is context again: who the teacher is is a variable that generally can’t be controlled for with reasonable sample sizes. Additionally, a educational research experiment generally can’t be blind, as both the teacher and the students know whether they are doing something different from usual. Given these limitation, saying to researchers “seek truth, but accept that you will be wrong,” may be the only way to move forward honestly.
Jason DyerJuly 5, 2011 - 2:04 pm -
This reminds me a little of Rosenblatt’s Nine Antirules of Journalism, if you’re into the contrary thing.
AmyJuly 6, 2011 - 9:21 am -
Labaree’s so much fun! I don’t know him personally, but I’ve read his work and his syllabi- they have a similar bite to them.
DaveJuly 7, 2011 - 7:18 am -
It’s not “be irrelevant, wrong, and lazy”, it’s “be more irrelevant, wrong, and lazy than your first inclination.”
Imagine research strategies lie on a spectrum between perfectionism and carelessness. Most researchers skew so far toward the perfectionism side (myopia, perfect truth, diligence) that if they actually aim for being a little more careless (irrelevant, wrong, and lazy), they’ll end up back in the middle of the spectrum.
The problem is that it’s a heck of a lot more complex than that. Sometimes, being a balanced person is the best way to move forward. Sometimes, being at the extremes is the best way to move forward. If you’re to the point of realizing that, there’s not a lot for you here in this article…which makes it seem like Labaree followed his own advice a little too far. ;)
Joe HendersonJuly 9, 2011 - 9:09 am -
From the Journal of Cell Science:
Relevant here, I think.
Suzanne GalvezJuly 11, 2011 - 8:24 am -
O.K., interesting reading. I got a kick out of the sarcasm. (Then again, I think Steven Colbert is hysterical…) Keeping an eye on the big picture, staying focused, working efficiently are all important qualities. I have been SO guilty in the past of taking off on some tanget that has caused me more work than I care to admit! When all is said and done I look at it and think “I didn’t even answer the question!” Back to the drawing board… It’s important to keep open to discoveries along the way. As for myself, if I can stay focused on my topic and sift through irrelevant information to glean the truly relevant information, I’ll be happy.