The Ed-Technologist’s Self-Evaluation

… the week is shaping up once again to be more about tools and vendors than about … the very essence of how teaching and schools are being pushed by the shifts that are occurring. – Will Richardson, an up-and-coming edublogger, questioning the momentum of the ed-tech conversation.

My word. If you people keep up this kind of clearheaded self-critique, this blog’ll have nothing left. It’ll digest itself, slip into a peaceful coma, and die smiling.

The Ed-Technologist’s Self-Evaluation

Are you about the tools or the teaching? Are you about the clenched-fist revolution or the game-changing evolution? Or, ideally, are you about both at the same time?

A scenario – only sorta hypothetical – that has nagged me going on eighteen months:

Kristi is a technologically-adept fifth-year Algebra teacher. She blogs, both personally and professionally. She can develop basic fluency in any tool – online, hardware, or software – you throw at her inside a week. She will let you do whatever you want to her classes, from the lesson plans on down to the seating arrangement. Hell, for the sake of the hypothetical, I’ll spot you laptops – MacBook Pros, XOs, Asuses, or whatever – on every desk.

The only requirement – and this is as minimal as they come – is that you cover her state’s Algebra standard to the same extent she has these last four years.

Do You Have A Plan?

I’m not asking for a year’s worth of lesson plans, a curriculum map, a first-day activity, or even a raised hand. I’m asking to ask yourself, “stripped of all my usual impediments and foes, do I know how to help this teacher?”

And if you don’t, I’m going to suggest here that you’ve driven yourself to distraction with what’s new *coughs in Plurk’s direction* and lost sight of what’s useful, that you’ve confused your adult enthusiasms with your students’ needs, that you’re approaching the learning transaction from the new tool downwards (“Plurk is awesome. Where can I fit Plurk into the classroom?”), rather than from the necessary instructional goal upwards (“What is the best tool – offline or on-, new school or old- – for teaching this concept?”).

So much of the ed-tech conversation has been motivated by, written in response to, even defined by the School 1.0 boogeyman. I mean, if I want a thousand words of big-picture idealism, of reductive analogies between traditional schools and modern jails, of rabblerousing, of browbeating, I know several hundred places to look.

That’s all fine and fair. The boogeyman exists, after all.

But the Kristi’s exist too. And they’re getting restless.

Don’t ask me how I know.

photo credit: Ewan McIntosh

About 
I'm Dan and this is my blog. I'm a former high school math teacher and current head of teaching at Desmos. He / him. More here.

46 Comments

  1. Hi my name is Kate, and I am a Kristi. (except I don’t have a blog, because I have nothing original to contribute). But Dan, you shouldn’t worry about us so much. We have a quick and accurate bullsh*t detector. If it doesn’t help deliver the content to the chillens, we’re not interested. We have walked out of inservices during the first break, we have a select maybe dozen edu-feeds in our RSS readers (and purge them frequently), and we are ruthless in selecting what we let into our classroom, and allow to stay. We cherrypick and are doing good things. Chin up, young person.

  2. As usual Dan your post brings to mind all of the “fun” questions I can pose to whomever presents at our next “tech” PD. :) Always enlightening to visit…

  3. Hi Kate, not worried about you or other Kristi’s at all. I’m saying they’re ready for more than the usual School 2.0 plaint that they need to get with the times and get a Flickr account (or whatever).

    And whoever told you you have nothing original to contribute to this conversation was lying. I’ve read your comments.

  4. Two points:

    To the extent that educational technology experts take math seriously, they are going to have a bias toward questioning the content of your math standards, because computers and computation change the nature of mathematics in a pretty fundamental way.

    But, I think the people you’re thinking of not only don’t think about math much specifically, they take great pains to avoid the reality of academic disciplines entirely. It isn’t just math. As you are pointing out, this renders the practical value of most of their ideas to be quite limited in practice.

  5. Dan – Interesting. As I sat here today playing with plurk (yes, I admit it), watching the NECC streams, and working through the lessons for my new prep next year I was writing notes to myself on concepts that will need to be supplemented and trying to write decent assessments.

    Did I come away with anything to use in my classroom next year? Nope. Will I? Possibly. But as of yet I haven’t found anything that is going to revolutionize (or lead to evolutionary changes in) my pre-calc classes. Right now my tech plan for my cherubs includes graphing calculators and Sketchpad. And my projector – of course.

    Kate I agree with Dan. :)

  6. Dan – I’ve decided your next job needs to be in an Elementary school classroom. I just happen to have an opening in 5th grade. San Diego is a great place to live. Our school is 30 minutes from La Jolla Cove and you just can’t beat the weather. BTW, once I actually post on my blog more than once a month, I might consider checking out Twitter, but probably not!

  7. @Tom, the subtext I’ve noticed around here w/r/t academic disciplines is that the only ones worth teaching are the ones which proceed directly from Web 2.0 / read-write technologies, etc. etc., ’round and ’round. Kind of a bummer.

    @Jackie, not saying there isn’t value to Plurk or Mogulus or whatever it is we’ll have forgotten in a month, I just like that you’re looking first to your instructional goals and then upwards to cool new tools.

    Can we note also, perhaps for Kate’s sake, that a year ago you ran basically the same timid line about blogging? And look at you now. Twitter followers. Subscribers. Technorati authority. Basically a blogging juggernaut. That’s fun.

    @Dan: I’m very interested in living somewhere in California that isn’t on fire.

  8. To paraphrase “Zen and the Art of Motorcycle Maintenance,” we have to make sure we start asking “What’s good,” rather than always asking “What’s new?” “What’s good?” is a better, more meaningful, more Quality question.

  9. Hi dan!

    I wanted to post a general “thank you” for blogging. I picked your blog as one of only 4 to put on the blogroll of my blog about being a new math teacher: http://sinesoflearning.blogspot.com/

    I’m reading through your blog from the beginning (right now I’m in January 2007) and I’m learning a lot from your thoughtful entries. I was especially pleased to note that your concept checklist form of assessment was something I was thinking of doing with AZ’s state standards, so now I know it is possible! Thanks for blogging!

  10. OK re-reading, I didn’t mean for that “nothing original” comment to come out so emo, but thanks for the encouragement. Just that I don’t have enough to justify jumping that far down my list of things to think about maybe doing someday.

    Funny story, when I first read this post I assumed “plurk” was a fake placeholder name, like “foo”.

    Dan Winters – I know you’re being a tad tongue in cheek, but I’ve been thinking lately about root causes, and chickens and eggs. As an aside, the cut score to “pass” our new Integrated Algebra exam in NY was 30 out of 87. How can we help elementary teachers do a better job with math? (I’m sure everyone knows an awesome elementary math teacher, but let’s be real, lots of them hate and fear math.) But doesn’t that go back to the kids who become elementary teachers having to learn math better? But where are their teachers going to come from? Aaaaargh.

  11. Dan,

    Good post. Brave man you are to take on the “establishment” education bloggers. I was taught by Will in a tech class for my doctoral degree. A bright guy with the right intentions.

    With regards to NECC, I watched some of the live video. Wasn’t interested in much of it. Maybe because I have heard Will in a much more intimate environment.

    You’re right on point – we have to consider what is applicable, not just cool, for our classrooms and our teachers. As a high school principal, I must be very careful not to expose the staff to fads and instead get them to see palpable tools and how they can be relevant and HELPFUL in their teaching. Much of what I read and see is based on coolness, not on using what works for state and district standards.

    Great post.

  12. @Mike, for whatever it’s worth, I’m not taking on Will, whose reflection on NECC I find admirably solid.

    It is interesting, though, how “establishment” is kind of a relative term. Relative to education bloggers, posts like these exist on some outer fringe. Relative to education in general, though, I’m advocating for the status quo.

  13. So maybe it’s because I had a really long night at Yankee Stadium (I’m a Mets and Sox guy, so you see what I’m willing to do for my ladyfriend), but I can’t figure out what, exactly, it is that Kristi needs help with. Her students are meeting state standards, and she, according to your scenario, can teach them how to use whatever technology she wants.
    You want us to help her with something, but I’m not clear on what she needs help with. I know you *never* construct straw men in your blog posts, so help me out here.
    If the tech isn’t going to improve the teaching and understanding–or, at the very least, the day-to-day classroom management stuff–then it’s not going to help. Teachers who blog don’t necessarily demand that their students blog (you know this, and I learned it the hard way a couple years ago).
    Of course, I just walked right into the point you were trying to make with this.
    It’s time to leave the Web 2.0-or-crappy-teaching people behind. You’re wasting bandwidth by even giving them the time of day. Let’s move on to this issue:

    What is the best tool – offline or on-, new school or old- – for teaching this concept?

  14. I wouldn’t dismiss the technology initiative entirely. There has been a lot of good that has and continues to come of it. But I like what Chris said about asking “what’s good” rather than “what’s new.” That’s the key, I think.

    As a new teacher, and being of the age where I have pretty much grown up with technology, I struggle pedagogically on when to use technology and when to forgo it.

    A small but significant example: About halfway through the year, I let my 9th grade (world history) advanced students use our classroom set of laptops to take notes from lecture. I didn’t notice any blatant classroom management issues with this, but I could tell their attention on me and the information I was presenting was being taken over by their overwhelming need to IM each other and browse the internet. I just can’t compete with that.

    So next year I am not going to let them use laptops to take notes. Some would say, “Well if they want to play games during lecture, it’s their grade.” True, but at the same time, I also lost a lot of discussion and interaction in my lectures b/c they were hurriedly typing the notes WHILE “goofing around” on the internet.

    But I feel pressure to use those laptops every day, so maybe that is why I felt compelled (and might still) to let them take notes instead of ol’ fashioned pen and paper.

  15. @Jeff, I can’t leave ’em alone. In the two years I’ve blogged I’ve never scabbed up to how clueless some edtechbloggers are to what happens in the core academic classes, much less in a math class, how they’d rather immolate the whole thing and build it from the ground-up (presumably) from social networks than work within that one constraint.

    That said, I never said I’m doing as well as I (whoops – Kristi. I meant Kristi.) could by her state standards. She’s so open to a set of substituted techniques yielding the same instructional value (or better – let’s aim high) using 21st-century tools.

    So I dunno. Until the conversation in my reader starts subsuming a little more humility in the face of these huge challenges, I’m afraid you’re gonna have to tolerate these quarterly screeds.

  16. I find your comments very true, Dan, but I cannot help but think that the more pressing issue is how many teachers actually have access to these tools to even become “over-hyped” with the fads.

    Me, I’m lucky if I have a LCD projector available daily, much less my principal presenting me with a new technology.

  17. Dan, I’m on your ship with you. And I don’t think thoughtlessly adopting some tech thing is a good way to go.

    But I can’t help but think that there is something to be said for that teacher who is discovers some plorf (or whatever) and gets excited about it. And yeah, maybe part of the motivation is that s/he wants to figure it out. Because it’s new and cool and s/he has figured out how it could be used in the classroom. And wants to try it out just because.

    If they’re young (like me), they won’t *know* if it will enhance student understanding. But they suspect it will (because they got excited because they saw how it could relate to some part of the curriculum), and there is no real way to know without trying it out. Our classrooms are our own personal laboratories and we do experiments hourly. Someone needs to be pushing the envelope, mutate some newfandangled tech tool, and see if it works out. And those people are likely going to be the teachers who are fascinated with it and have the time to keep up with it, and get excited by it.

    I guess what I’m saying is that I’m starting to wonder if the dichotemy really is:

    “Plurk is awesome. Where can I fit Plurk into the classroom?”

    v.

    “What is the best tool – offline or on-, new school or old- – for teaching this concept?”

    It seems a bit false to me. I suspect for most teachers who read your blog that it might be a little bit of both, some experimentation with some reflective work, that seems more accurate.

  18. sam shah, I’ll cop to a certain mix of priorities and I’m specifically trying to dodge false dichotomies with:

    “Or, ideally, are you about both at the same time?”

    I’d only add to your “it’s a little bit of both directions” critique that failing from the top-down direction (ie. You really wanted to use Plurk. You hamfisted it into a unit on quadratics. Kids were lost.) is a little more tragic and smells of hubris a little stronger than the alternative (ie. “Okay, we’ve gotta figure out quadratics. Have to. Maybe Plurk has something for us here.”).

  19. If you look at technology as technology, you miss the higher level potential. Instead of getting hung up on the technology think of the whole porridge as multimedia delivery systems.

    We’re in a time which is a lot like the changeover from story telling and court jesters to moveable type. The problem with technology application in education is that you’ve got this amazing race track with no cars to run on it. We can move from books and classrooms to multimedia communication with one on one council but it’s hard to break out of our 19th century models for education.

    If an individual teacher tries to take advantage of it all it sucks up amazing amounts of time and it’s educational value can get hurt by this time sink. Until we find a way to get teaching to be other than this flat one layer delivery system, we won’t have the specialized focii it requires to leverage the multimedia and as a result the technology stays stuck on technology for technology’s sake.

  20. Dan,

    Chris Lehmann has already stated it better than I but a quick amen to your statement of teachers and tech integration specialists losing sight of what’s useful.

    What’s new? Plurk

    What’s useful? A trained teacher picking the right tool for the job

    glennw

  21. Hey Dan… just one thought about humility. Any teacher, principal, etc… who is not utterly humbled by the enormity of the job in front of us all is going to make serious errors of hubris, and s/he’ll miss a lot of what this is all supposed to be about. So yeah, I agree.

  22. Chris, I was reading Ewan’s notes on your preso, which invoked a lot of the same language I was aiming at here, which reminded me of the weird synchronicity between our Faculty Room posts, which, overall, struck me as weird. How far is SLA from Santa Cruz again?

  23. So let’s use tech for what helps us teach, and what helps us teach is Dan’s great assessment system (mastery grading, test one skill at a time, no passing till you master it, unlimited re-tests…). But that can be a lot of paper-shuffling and gradebook work.

    So let’s automate it. When a student is ready for a test he hops on a computer, gets a computer-generated randomized version of the test he’s ready for, and the results go on-line for the teacher to see.

    Does anyone know good software for that?

  24. I have little to add here but feel the need to nod in agreement.

    Most definitely we need to differentiate between what’s fun for us and what makes us better practitioners. I’m unapologetic about playing with Plurk (or Twitter, or Tumblr, or [insert Web 2.0 tool o’ the day here]…), but I don’t pretend that any of this stuff has any educational value whatsoever.

    It’s fun for me.

    When we start mixing our hobbies and interests into our professional practice, we’d best not be kidding ourselves as to the real value we’re adding to the classroom or school [“Now kids, I don’t want you raising your hands to answer this question – that’s so School 1.0. Yes, I know I’m standing right in front of you, but if you think you have the answer please @ me on Twitter!”].

    I cringe to think some are wrapping lesson plans (or staff development plans) around the latest and greatest web tool just because it’s new and shiny.

  25. Dean –

    A combination of Examview and Blackboard would do it – both costly systems so possibly not practical if your district doesn’t already have them – interesting suggestion…I think you’d have to still give your whole-class assessments on paper, though. At least I would. But I’ll definitely be considering the logistics of this for follow-up assessments! Thanks!

  26. Just to follow up on my earlier comment, I did find sessions that directly relate to my teaching next year. Chris Lehmann’s session (Understanding by Design is my next read) and David Jakes’ & Dean Shareski’s session on PowerPoint. And I’m now reading Mindstorms by Papert (who’s name came up somewhere).

    And yes, a year ago, I was basically in the same place Kate is now. What are you waiting for Kate? As for the rest of the stuff: ranking, followers… yeah. Not quite why I’m doing this. :) Although I should really write some of my thoughts on my own blog soon.

  27. Kate,
    Thanks for suggesting Examview. It seems to have what I need, including a question bank for the textbook we use. (With a question bank the teacher can make unlimited randomized versions of tests so the kids can take them as many times as they need). And the price is fine ($139 for a 1-teacher license). If the district wont buy it I may take it out of my allowance.

    Kate, here’s a question: Why do you think whole-class assessments need to be on paper? The only advantage I can see to paper is so students can show their work and the teacher can see their thinking. That’s important, but how about students turning in a work page for 1 or 2 of the online test questions? The teacher can review the work there, but not slow down the (automated) test-feedback-retest cycle.

    Dan,
    In your (great) assessment system, have you moved in this direction or are you using paper?

    Too many parens in this comment (dont you think?)

  28. @JackieB, NECC seems far too big and intimidating for an introverted creature like myself. Maybe if a, uh, surrogate mother figure was there also, I uh … um … weird.

    @Dean, I’d love to have some sort of auto-generative testing thing running. My only two reservations are:

    1. It has to align with the state standards and my own.
    2. I have to be able to print up one concept quiz on-the-fly, whenever I want, for those kids who leap into the middle of my day looking to drop some knowledge on me.

    Not sure if those are too much to ask.

  29. For me, giving all assessments on a computer would be impractical. I need them to be done in class (not at home) so that I know someone else isn’t doing it for them. At my school that would involve rolling the mobile laptop cart into my class every time I need to assess, with the attendant 5 minutes lost at the beginning and end of the period to checking out and turning in laptops.

    Also, you bring up a good point about showing work. I’d rather have everything on one page than have to keep track of a “work page” that goes with their online test. Typing equations and diagrams into a computer is tricky, and I think the students have enough to worry about. This is actually one of my hangups about using blogs or wikis in class – I don’t want to take time to teach them MathType or LaTex and don’t want to try to decipher things like f(x)=x^3+3x^2+3x+1.

    Anyway Examview has its drawbacks but it’s still one of the best test generators I’ve seen. Maybe they will send you a preview copy or trial version to check out. My biggest criticisms are that the question bank for my textbook kind of sucks and I have to write many of my own questions, and it’s difficult to format to make it look functional and pretty.

  30. Dean,
    You make some interesting suggestions for a “test-feedback-retest cycle,” but I’m not entirely sure that that’s what you really want. If you have an online test that’s a fast, automated test/retest cycle, you’ll end up with kids that happen to know the answer to the questions they’re asked because they were given the answers in their feedback section, but kids who don’t actually know how they got the answers they did. This is especially true in some of the non-math online tests, where the questions may be randomly generated from a bank of (at best) twice as many questions as are on any given test. (And as for math, I agree with Kate, typing in math is confusing for many students. Our physics online tests tried to make students learn a simplified LaTeX, and that caused at least a little confusion.)

    While I agree that perhaps paper isn’t the way to do it, that does tend to slow down the cycle somewhat, allowing for questions, and analysis. As for whole-class assessments, those tend to end up most easily on paper because of the relative scarcity of computers to paper, though I do agree that the faster feedback afforded by computers can certainly be a benefit in many cases.

    You ask about “students turning in a work page for 1 or 2 of the online test questions,” which can certainly be a reasonable idea, but requires the teacher to not just assume that because the answer was right (on the computer) the student actually understands all of the material. And if your computer labs are as crowded as ours, finding a place to put the paper to write on could be difficult in and of itself. (Though that’s a problem with a different solution.)

    So, I agree with most of what you’re saying, I just see some problems in actually implementing them. That’s not to say they’re not good ideas (they are!), or that the problems can’t be worked around (I think they can), just that they should be considered when moving forward.

    (Too many parentheses? Bah.)

  31. Dan, again with the mother figure bit Dan? Hrmph.

    And I wasn’t actually at NECC.

    Dean – my concern with the online test generators is that the one’s I’ve seen produce questions with varying degrees of complexity – allegedly for the same skill. Also, when working from screen to paper some students who have tracking difficulties can too easily make mistakes (mixing up parts of one problem with another comes immediately to mind). Yet another problem I have with the online questions that are multiple choice is that the distractors are badly written.

  32. Dan, Kate, aschmitz and JackieB,
    Thanks for your comments. You raise a pile of practical issues which I’m grateful to face now rather than suffer in Sept. I ordered the 30-day trial of Examview and I’ll let you know whether any of the practical problems are deal-killers.

    To start out slowly this Fall, I may set up a series of mastery quizzes (Dannish, one concept at a time) which would run parallel to a series of less frequent paper-and-pencil unit tests. But….if and when the technology catches up to the idea, I’d like to switch all assessment to the computer. That’s the only way I can see giving the kids the benefits of self-pacing, mastery grading & unlimited retests without driving the teacher up the wall with grading.

    Thanks again for getting into this w me. I’ll let you know how it goes.

  33. This seems to have turned into a referendum on how to test, dan style.

    Which means this is probably a good time to ask this question (especially since my two new preps for next year are the same ones dan teaches):

    How do you come up with one representative question for each concept? What constitutes mastery, and how do you capture that with a single question? More importantly, how do you devise a question that is new and different for their second, third, or eighth tries that still tests the same knowledge at the same level?

    I’m interested in rolling this into my teaching, but I always seem to need a battery of overlapping problems to evaluate my students’ knowledge. I’m pretty sure that pulling one problem out of a textbook quiz bank isn’t going to cut it.

    (btw, dan, this is a poorly veiled hint at doing a post where you discuss this issue, including what you perceive to be the difference between your first question, and the subsequent mastery ones.)

  34. Wish I could elaborate this into a full post but the methods are so specific to each concept.

    I mean – stammering here – I just make ’em more difficult.

    If we’re graphing lines in slope-intercept, and we started our practice with whole number slopes, I’ll give ’em one with a fraction, and then I’ll give ’em one with a negative fraction, and then a negative whole number, and then when I feel like they’re starting to game this thing, I’ll reverse the order (y = -2x + 5 becomes y = 5 – 2x).

    It goes on and on.

    Kids will come in eight times for a single concept and when they finally land the perfect score, only rarely do I feel like they just wore the concept down. (And those are the concepts that usually don’t show up again the next year.)

    The reason they come in eight times is the same reason they don’t get frustrated which is the same reason they keep chasing after the skills is because it costs them nothing.

    They can only improve.

  35. Excellent question. As someone who teaches both technology and other subjects, I have a tendency to drift towards over-hyping technology… However, some of the foray into the world of hype is an attempt to leverage a pre-existing technology (i.e. cellphones). I think in cases where it’s a ubiquitous technology, it’s valid to ask how it can be used in a class, rather than trying to find a separate technology to answer a need…

  36. First and foremost, cellphones. I spend far too much time telling students to put their phones away when we could be using them to look up information, answer questions, or something else.

    With pervasive wireless, gaming systems like the Nintendo DS and the Sony PSP become tools for online research just like any laptop.

    These are already in my classroom, just waiting to be used… if only I could find some educationally appropriate use for them.

  37. Ian H., while some students certainly do have cellphones or other wireless devices (I had an XO in my backpack most of last year…), a good number of them don’t have anything like that available. Have you come up with anything that would let them participate as well? I’ve thought about it before, but the only solutions I’ve come up with are having a (partial?) classroom set of something and sharing those when necessary, or working in groups.

  38. I had the same concern, Andy, and basically concluded that no one’s ever going to have 100% of everything. (I can’t even trust my kids to bring a pencil or pen every day, for instance.) So differentiation is the name of the game. Great stuff for everyone, whatever’s in their pockets.

    Easy, right?

  39. Wish I could elaborate this into a full post but the methods are so specific to each concept.

    then perhaps a series of posts, as you cover new subjects? please?

    it seems all i have is more questions, but even if the examples don’t answer them perfectly, i’m getting more and more ideas.

    no one’s ever going to have 100% of everything.

    was it you that mentioned that magic differentiation powder?