2011 July 8: Labaree just sent our class this same sermonette only now edited for publication. So take your pick. Polished or off-the-cuff.
This is a transcript of the â€œsermonetteâ€ David Labaree (author of two of the best papers I cited last week) delivered last week at the end of Stanfordâ€™s spring quarter proseminar. He gave me permission to share it with you. The tl;dr version is this:
- Be irrelevant.
- Be wrong.
- Be lazy.
Hell of an enticement, right? Iâ€™m posting it primarily for myself. Iâ€™m positive Iâ€™ll revisit it and find it intriguing in different ways the more I work in this field.
June 1, 2011
Be irrelevant. Even in a field where the problems of the field are so important and so demanding, the argument that Augier and March (2007) make is really smart. They say, on the one hand, the push to be relevant causes two kinds of problems. One has to do with what they call myopia and the other is ambiguity.
The myopia problem is that youâ€™re looking at something thatâ€™s presented to you. Fix this. And so you burrow into this. But this is typically located in time and space. And the circumstances are such that you pull it in close. (Thatâ€™s why he calls it myopia. Itâ€™s a nice image. You pull something up close if you have myopia. Youâ€™re nearsighted.) And so in the process of looking at it up close, the whole context disappears. And you start treating the problem as though it were not connected to that context. And then you often end up engineering a change that may or may not work in that setting but is not transferable because itâ€™s actually involving things that are contingent on a particular context â€” a particular time and a particular place â€” and efforts to then generalize from that show that the insight was actually pretty irrelevant in terms of workability.
Also in terms of time. If youâ€™re burrowing in too much on fixing the particular problem, by the time the work gets out, that problem has already evolved into something else. That was then, this is now. And even in that setting it may not work. It has already been outgrown because you are missing the evolutionary component of whatâ€™s going on.
So thereâ€™s a certain sense in which relevant research may actually have a very short shelf life. It may start to smell bad after awhile. It may have to have a buy-by date on it that says â€œAfter this, it may not be good anymore. It may not apply in any other location.â€ So in some ways, not trying to be so relevant may actually come up with insights that are more transportable, more useful, and are actually more applicable, even though that wasnâ€™t your intention.
The other issue they raise that makes me say â€œbe irrelevantâ€ is that relevance is kind of a rhetorical plane and one of the things you have to say is â€œrelevant for whom?â€ Is it relevant in the schools? Is it relevant for teachers? For students? For administrators? For superintendents? For policymakers? For politicians? For parents? It depends. Maybe whatâ€™s relevant for one is not good for the other. High-stakes testing is highly relevant for policymakers in order to make the claim of accountability in schools. It may be very harmful for teachers and students. So itâ€™s relevant. Research supports it but it may be a relevance that depends very much on a relevance â€œfor whom?â€ Thatâ€™s a claim thatâ€™s not generic. It has to be established but itâ€™s not generalizable.
Thereâ€™s also the relevance â€œfor what?â€ For what end? What are we trying to accomplish in schooling? Are we trying to make better citizens or more productive workers or help people get ahead or reduce social inequality or what are we trying to do? Well, the relevance of the research depends on the relevance to which of these claims itâ€™s focusing on. One argument is that in some ways itâ€™s a foolâ€™s game to try to be too relevant in a field like this and, counter-intuitively, the most useful research may be the stuff that doesnâ€™t seem to have an immediate application when youâ€™re actually doing it. And an effort to be slavishly useful may give your work a limited purview and a very short shelf life.
All right. One piece of advice that nobodyâ€™s going to follow: â€œBe irrelevant.â€ Another one nobodyâ€™s going to follow, and thatâ€™s â€œbe wrong.â€
I think one of the dangers in programs like ours is that we encourage people to find answers, to be right, and that makes you risk averse. And my argument is that itâ€™s much more useful to be interesting, and to provoke thought with your ideas, even if youâ€™re wrong, than it is to be right in a manner thatâ€™s not very interesting, not very provocative, and not very likely to spur anyone else to do anything.
You never establish claims for all time. Truth is an ideal you pursue but you never reach it. And if you ever waited to nail down everything before you published something, you would never publish anything. Whatever you do, you have to recognize itâ€™s going to be a partial statement. Itâ€™s going to be at best a partial truth. Itâ€™s something thatâ€™s true under certain circumstances and under certain conditions and with certain limitations about it and thatâ€™s enough, actually.
Youâ€™re not in the position where you need to make everybodyâ€™s counterargument to your argument. You just need to make your argument effectively and say to yourself, â€œIs this something thatâ€™s not in the conversation that should be in the conversation? If so, I should get it out there. And I can picture what some people will say in response but I donâ€™t have to make that. Let them make that. I want to make a strong case in this direction. I donâ€™t want it to be easily dismissed or laughable. I want it to have solidity, validity, and rhetorical effectiveness but itâ€™s not my position to find out what the absolute final truth is because thatâ€™s not findable in my lifetime.
â€œSo Iâ€™m going to be part of a conversation and the conversation is what matters and Iâ€™m going to learn from the conversation and in the process Iâ€™m going to revise what I did and Iâ€™m going to admit that some parts of that were wrong and Iâ€™ll move ahead and Iâ€™ll publish something else that is also a contribution to the literature. It helps with the conversation, but it also has plenty of possible responses to it. Iâ€™m going to learn from those responses too and Iâ€™m going to continue working on this in a somewhat new form after having acknowledged that certain parts of what I did before were not that good.â€
Thatâ€™s okay. Thatâ€™s actually considered a successful career. Thatâ€™s doing your job as as a social scientist. If youâ€™re trying to nail it down and be right you probably wonâ€™t publish anything. Youâ€™ll keep waiting until you get it right. It has to be good enough to be provocative. Research is a provocation of thought. If youâ€™re provoking thought with people and if youâ€™re giving them a slice through a situation thatâ€™s a little different, that makes them think and reframe their understanding of something in an interesting way, thatâ€™s a successful piece of research. Even if itâ€™s wrong, in a lot of ways.
As you know, itâ€™s very easy to take even the best study and trash it. On methodological, theoretical, or other grounds. So thatâ€™s not the test of a good study. The test of good studies is, did it have an impact on you? Did it provoke your thinking? If so, itâ€™s worth doing.
Be irrelevant. Be wrong. Third one. Donâ€™t tell your advisor about this, but â€œbe lazy.â€
Thereâ€™s a real danger in educational research that you just plow ahead. â€œIâ€™ve got a big pile of data in front of me. Iâ€™m just gonna wrestle with it. Iâ€™m gonna run this data set using every single test in all of the statistical programs Iâ€™ve got and keep plowing ahead until Iâ€™ve got every permutation. Iâ€™ve got all this qualitative data. Iâ€™ve got all these tapes. Iâ€™ve got interview tapes. All these other kinds of things. So what I need to do first, of course, is to transcribe everything and then to code everything. And then try to put it all together.â€
No â€” donâ€™t do that. Donâ€™t do that. You want to transcribe very little of it. Most of itâ€™s garbage. Most of itâ€™s noise. You were there listening. You know what was in there. You donâ€™t have to transcribe every bit. Same thing with a big data set. You shouldnâ€™t be wallowing in the data hoping itâ€™s going to speak to you. It wonâ€™t. You have to make it speak. Youâ€™re looking for the music in the data. Most of whatâ€™s in data is noise. So your task is not to somehow encompass the whole thing. Itâ€™s to find a strategic route through the data that provides some kind of insight thatâ€™s not out there in the literature right now.
And often that means not being diligent. Diligence can be a dangerous trait in a grad student. It means â€œIâ€™m just plodding ahead day every day. Iâ€™m going through another test. Iâ€™m transcribing another tape. Iâ€™m doing research.â€ No. Youâ€™re not. Youâ€™re transcribing tapes. Research means youâ€™re actually trying to figure something out, youâ€™re thinking your way through it.
Shortcuts are very nice. Shortcuts. â€œDo I have to go through all these data? Maybe only some of it matters. Maybe some of that whole issue is over there.â€
I spent two years of my life working with a quantitative data set that I generated, coded, keypunched, analyzed, and had print-outs coming out my ears and it ended up when I published the book, I had a colleague, David Cohen. He looked at the book and said, â€œAll of the data you had in there seem to be a footnote to the claim, â€˜Central High School had meritocratic achievement.â€™â€ Two years of my life. A footnote. It turns out it was an interesting finding. It was counter-intuitive. But the actual interesting stuff was elsewhere in the data that didnâ€™t take me two years of my life plowing through all of the stuff.
So donâ€™t ignore the low hanging fruit and donâ€™t assume that the only way to get from here to there is the longest possible route through the most amazing morass of data. Itâ€™s okay to think your way through and around a problem. Thatâ€™s a good thing to do. Sometimes you find something and youâ€™re gonna have to plow through it. But you want to have some confidence that youâ€™re doing it for a good cause and youâ€™re not just doing it in a kind of Stephen Colbert way. Itâ€™s â€œresearchiness.â€ Researchiness means â€œI need to analyze data. Itâ€™s what researchers do. Give me some data. I need some more data.â€
No, youâ€™re supposed to come up with something interesting to say and it may be that only a little piece of data are actually germane to that and it may be that itâ€™s an entirely different data set way over there that you want to be working on so why waste your time on this.
So as I said file that way. Never follow any of this. Donâ€™t tell your adviser about this. But you might want to keep it in the back of your mind as a kind of cautionary tale about how you donâ€™t want to get caught up in the aphorisms and the common senses of what research is. You have to keep in mind, â€œWhat am I doing this for? What am I trying to do here? What am I trying to get out of this? And how can I go about doing that in a way thatâ€™s productive and not just busy?â€
Augier, Mie & March, James G. (2007). The pursuit of relevance in management education. California Management Review, 49(3) (Spring), 129-146.