The MTT2K Prize

Let me just point you to Justin Reich’s post on The MTT2K Prize he and I are co-sponsoring and co-judging. I only want to add a +1 and maybe a smiley face next to this sentence:

As far as I’m concerned, MTT2K has brought all kinds of good to the world.

I’d like to see some more of the kind of engagement we saw this last week, the kind where online criticism turns into improved outcomes for millions of students in the span of 24 hours. I’m excited to see what comes of this.

I'm Dan and this is my blog. I'm a former high school math teacher and current head of teaching at Desmos. He / him. More here.


  1. I think this is a great idea, and thank you for actually investing in this! No doubt it will effect some very positive changes, especially in Sal’s earlier videos.

    However, I disagree that the original video was “poor lecturing”. He very clearly outlined the 4 possible situations when multiplying two numbers, and what the result will be. His goals when he originally made that video were merely different than what some people WANTED his goals to be.

    I think there’s a fundamental difference in what some people THINK Khan Academy is versus what its mission actually is. Khan Academy is no short-term project, and more importantly, it is entirely open source and largely community driven.

    It really disappoints me to see some educators attack KA as if it’s supposed to be a finished product. The kind of criticism in MTT2K and this contest will absolutely be useful and positive, but talking about “KA should be grateful to these two guys… they gave up their time for free proofreading and peer review” or “Oh hey I hear he’s Bill’s favorite teacher, LOL” does nothing to contribute to the discussion.

  2. Mark:

    However, I disagree that the original video was “poor lecturing”.

    Khan disagrees with you. He pulled the video and amended it.

    Any questions or commentary about the MTT2K prize is fair game for this thread. General criticism or praise of Khan Academy belongs somewhere else and will be deleted here.

  3. psst…James Tanton’s name is misspelled on Justin’s post. Posting here since I have to login on the other site.

  4. mr bombastic

    June 23, 2012 - 5:22 pm -

    Using Khan’s videos is a nice way to work around some of the cultural and time obstacles that prevent more US teachers from doing lesson study. It would be a huge help if we could get more teachers thinking about how seemingly small details, in seemingly run of the mill lessons, impact their students understanding.

    I would like to see this type of analysis applied to other forms of instructions as well, but videos are in short supply. That said, there are many teachers that have an approach fairly similar to Khan. Perhaps Khan would not be the only teacher to benefit from these critiques.

  5. I’m looking forward to posting an entry!
    It’s very easy to sit back and complain or point out flaws, but it doesn’t solve anything. MTT2K is a great opportunity to be part of the solution or a way to help many students. Thanks for the opportunity!

  6. Posted at Justin Reich’s blog:

    Love it!

    Must say, though, that I’d take Jim Tanton at his worst over Sal Khan and his 8th grade teacher at their collective best. Dr. Tanton has extensive experience both as a high school mathematics teacher and in the Math Circles that Bob and Ellen Kaplan run in Cambridge, MA. I remember asking Bob Kaplan in 2002 when I went to see him run a session with math majors, graduate students, and professors at Northwestern University’s math club, where do you get teachers to work for you? After telling me about Jim Tanton, he said, “Finding people in Cambridge who know the mathematics isn’t a problem. The problem is finding people who know the math that can keep their mouths shut.”

    Somehow, I don’t think Sal Khan would have the first clue about keeping his mouth shut.

  7. I find the almost fanatical attempts to fend off criticism of Sal Khan and his product bizarre. He’s put himself in the position of “revolutionizing” mathematics education (and education in general). I’ve yet to see him beg off on all the praise that’s been heaped upon him. He’s spouted off on his educational ideas and his views of “the competition.” He is now, truly, a public figure. And so from any angle I can think of, he’s fully deserving of criticism, as is any and every example of his work.

    I do not find the retort, “Can you do better?” a valid response to a critique. It is not the responsibility of those doing critical analysis (and the word “critical” here does not have to mean “negative commentary; if it does to any given person’s mind, however, I hope such readers will substitute the more neutral “evaluate,” “evaluation,” or “evaluative,” as appropriate), to come up with a better product, though that option is open to them if they choose to try. Literary critics are not expected to produce better works of literature than those they analyze, or, in fact, any works of literature at all. True, the late American poet, Charles Olson, wrote CALL ME ISHMAEL and created a critical and biographical work of such literary brilliance as to perhaps have defined an entirely new genre. But there aren’t a lot of Charles Olsons out there. If critics have to be able to “do better,” before they’re allowed to weigh in, that would be a VERY convenient way for Sal Khan and his big fans to minimize the negative commentary that continues to grow regarding his work. But guess what? They don’t get to make the rules.

    I think doing a MSTie send-up of Khan is perfectly reasonable. I expect that there will be some excellent things coming out of this challenge. If no one who offers up a critique goes on to create a “counter-video” that gives a concrete example of how to do better, that won’t matter one bit. The profession of mathematics teaching will still be well-served by starting the conversations that no doubt will ensue. And others will learn from those dialogues, leading no doubt to improved lessons, whether on-line, in classrooms, or both. And that’s what we need much more of. Mediocre how-to math videos are a drug on the market already.

  8. Yes, the MSTie competition will do a lot of good, but the problem is the image people will get from it of constructivists will be negative. Is the tone really worth it? I’m going to try and upload a more info-heavy and less satirical video that I think would serve both sides well without causing strife.

  9. Michael Paul Goldenberg

    June 24, 2012 - 11:09 am -

    Need one frame him/herself a “constructivist” to critique a KA vid?

    Does being polite generally mean one’s message will gain credibility or attention? Sal’s not all that polite about other approaches if you hear his talks. Again, I wonder how he’s gotten the leverage to set the rules of the public conversation about anything.

  10. Sal’s not all that polite about other approaches if you hear his talks.

    Not sure exactly what Sal has said or done, but was parody, satire, or swearing (in a recent MSTie video on acceleration) it? I can’t condone that. The contest will be judged on helpfullness AND humor? This suggest the ulterior motive is not to do some good, but to vent/rage. Compare the tone of the videos and blog posts to what Khan wrote to the Edweek blogger”We at the Khan Academy really appreciate the feedback from you and your colleagues regarding videos that could use improvement. We think we are just at the beginning stages in our quest to make a truly valuable resource for the world and are eager to improve as much as possible (especially our older videos originally made for my cousins). We do strongly disagree that we only emphasize procedural understanding at the cost of conceptual. We think it is important to do both so that students have access to worked examples and deeper conceptual understanding, intuition, and proofs.”

    The tone says it all, and on a practical level, we are doomed.

  11. FWIW, I appreciate the two of you keeping this conversation civil and on point.

    Two lines I wouldn’t mind reading more about:


    I wonder how he’s gotten the leverage to set the rules of the public conversation about anything.

    When you make the kind of sweeping claims Khan has (ie. “there’s no reason every teacher in the US couldn’t use KA tomorrow”) about something as important as education, you invite criticism and skepticism. It’s necessary. No one should be able to put those kind of warranties on any educational innovation without getting that kind of scrutiny. Whether the MTTK2 teachers are too glib or snarky in their scrutiny just seems like much ado about absolutely nothing to me.

    Doug Dahms:

    The tone says it all, and on a practical level, we are doomed.

    Who is “we” and how are we practically “doomed”?

  12. I spent six years working in three MA public school districts as a math/science curriculum specialist/coordinator. My job in all three districts was to introduce constructivist approaches and provide professional development for teachers. I was working with highly-trained, professional math and science educators.

    At one of the districts, the teachers wanted to standardize on a method for teaching the multiplication of integers that centered on a song that helped students memorize the “rules.” Apparently, this method was considered both engaging and effective. Over 90 percent of the teaching I observed in math classrooms was procedural. Most recent math textbooks include a brief conceptual introduction (usually section 1) as a nod to constructivism, but then quickly abandon any conceptual framework by section 2. This leads students to disregard conceptual introductions and to impatiently demand that teachers just “tell them the rules.” In districts that use constructivist math programs (I use that definition loosely since I don’t really see any out there), the general consensus is that, while concepts may be useful for above average students, struggling students or students with special needs should focus on memorizing procedures since concepts are beyond them.

    So the question for me is: Why have our attempts to introduce constructivism into math education failed so badly? It has failed not just with the general public, but with math teachers, curriculum writers, policy-makers, and students. Constructivism may be succeeding in isolated classrooms, but not on any kind of large scale (as far as I can see).

  13. Bit of hypocrisy here, no? Posts in support of KA get warnings of deletion since they are “off topic”, but posts criticizing it get complimented since they are “on point”. I hope at least my email found its way to your inbox Dan.

    Anyway, @David – The thing is that if we teach people just procedures, we’re teaching them stuff that is already worthless (well, okay, addition/multiplication still have some value) since computers can do it better. If special needs students can only understand procedural teaching as we are teaching it now, that means we need to get better at teaching it. What’s the point, otherwise?

    @Doug – I don’t think tone’s going to matter all that much. The people who have watched MTT2k, and those who will watch the results of this competition, are the ones who want to improve the state of education in any context. Doing so requires an open mind and enough of a thick skin to not take satire personally ;) Plus it might garner more entries and some deeper insight into what makes KA’s older approach good/bad.

    As far as procedural learning is concerned, does anyone have a research paper connecting procedural to conceptual learning of math? My intuitive sense is that learning procedure first makes understanding the concepts behind it easier, but that teaching the concepts first is much more engaging and interesting. Also, to be honest it feels to me like the conceptual aspects of many parts of math are just “one more thing to memorize” unless you actually plan to go into theoretical mathematics later on. E.g., you can develop an intuitive sense of why a negative times a negative is a positive without having a formal definition or go-to explanation. Incoming flames?

  14. Mark:

    Bit of hypocrisy here, no?

    No. Your last two paragraphs wobbled off the point of the thread over to general praise of KA. Critiques of KA that don’t center around the parameters of this contest will also be deleted.

  15. @Mark — I’m certainly not in favor of teaching math procedurally. I was just pointing out that right now, despite decades of trying to change how math is taught, the procedural approach is still prevalent. Most math educators in the U.S. have learned quite a bit about the importance of laying a conceptual foundation, but that hasn’t made much of a difference in the classroom. If our goal is to have a real impact on math education for students, then I’m not sure how MTT2k is suppose to help.

    Regarding your second point: I agree that applying a conceptual veneer to our current practices is ineffective. Most math programs already try to do that; and it is just another thing that students have to memorize. Basically, we are trying to provide students with a mental model for understanding the mathematics that they are learning. However, mental models are constructed over years. Students need to be able to build on them and test them in order to refine them and trust in them. When I open a unit with three days spent building a mental model, and then move on (and the teacher the following year introduces an entirely different mental model), the mental model fails to be useful and is just excess baggage. A conceptual approach needs to span not just lessons and units, but also grade levels.

    I have some example that I can share with you if you’d like to take this conversation private.

  16. mr bombastic

    June 24, 2012 - 2:35 pm -

    @David Ng, thank you for sharing your observations. The instruction I have seen in my district and my children’s district is also overwhelmingly procedural — despite both districts using texts that are popular with the constructivists for K-8.

    I think a lot of the reason that constructivism has not taken hold is that many education professionals and teachers have an extremely superficial view of constructivism. It would help if more people looked carefully enough to recognize that the vast majority of US lessons are essentially procedural — whether they are straight lecture, or have been slightly disguised by group work or investigations or brief questioning. Looking at the details of Khan’s lectures is a good start for taking a more nuanced look at lessons in general.

  17. @David Sure! I’m really curious about your experience in general as well, trying to push conceptual standards and definitely any insight you have into the question you asked — WHY it hasn’t worked.

    You can reach me at

  18. @mr Bombastic I like the idea of using KA videos to spark lesson study discussions.

    @David Ng I would like to see some of your conceptual models. Have you or Will you be posting them on your website?

    My thoughts on why constructivist education gets the short shrift is two fold:

    1st it requires a degree of understand mathematics conceptually and connected to real life that many teachers don’t have. (Illinois endorsement in middle school math is I think 15 semester hours which for some means nothing more than calculus) [There is also no requirement to review/relearn conceptual understanding of math as part of professional development]

    2nd professional support for constructivist education is usually based on collaboration something many teachers don’t seem to do very well.

    Finally, to get on the point. It is not necessary for humor to be mean spirited, some might even say it shouldn’t. On the other hand I think the original video needed to be a bit scathing to gain attention of a wider audience. (look nobody was paying attention to valid critiques so let’s make fun of the videos and show people there are serious flaws that should be addressed.) It has done that job. Future constructive criticism delivered with humor should result in positive gains.

  19. The MTT2K Prize is a great way to get people starting thinking about video instruction, and how good the Khan Academy videos are. I’m also interested in creating a more general framework for examining these kinds of videos.

    I’m looking around for a general system for coding instructional videos, and haven’t seen one yet. I’d hate to have to re-invent the wheel. I’ve looked at the ones designed for coding in-class instruction, but classroom instruction is such a different medium than video instruction that I can’t even see how to apply the specific coding methods to a video. The purpose of the coding would be to come up with ways of categorizing moments of an instructional video in certain ways, and trying to generate data about errors and/or omissions in instructional videos.

    I wonder also how well one can construct videos so that they act as support for inquiry, so I’d especially like to use a coding system which is flexible enough to handle a variety of pedagogical formats in a video. Videos that ask questions of the watcher, and require thinking are far superior, I suspect, than videos which provide “the” correct solution. I think a classification system would be the only way to test this assertion however.

    If anyone has heard of such a classification system, or knows of someone who has created one for themselves, please let me know.

  20. Videos that ask questions of the watcher, and require thinking are far superior, I suspect, than videos which provide “the” correct solution.

    It’s difficult to make a statement like this without context. Are the students in a situation where some outside impetus forces them to answer the questions?

    One of my frustrations making my own videos is how to convince students to actually attempt the questions. I tried in my logarithm video to include an icon at the same time as asking students to pause, but I have grave doubts any students really did so. Alternately, one could put a question at the end and have no answer provided in video form, but even given some students bother to attempt the problem how many will bother to check their answer?

    Your response might be “they need a teacher to remediate”, but the whole point of Khan etc. is for students to be able to learn things without a physical teacher present. Khan’s solution is to then include electronically graded exercises to repeat a particular algorithm. (There’s some experimental stuff to go with the Brit Cruise videos, but that’s a very small portion of the site.) Clearly something is missing here, and I have trouble simply accepting the statement “good self-instruction is impossible”.

  21. @Dan, in response to #15 – come on Dan, I’ve got to agree with Mark when it comes to comments like #6, which has almost nothing to do with the MTT2K Prize and pretty much just attacks Khan.

  22. I get the sense that since Dan Meyer has done nothing but criticize KA since the very beginning, this MTTK Prize is simply a way to criticize in the guise of altruism.

    “Look at the good we’re doing, helping the children of the world by creating overly sarcastic critiques of someone who had an idea and ran with it. ”

    Please. KA is a work in progress. If you can create something better, do it and let people decide which product they want.

  23. Mansoor:

    I get the sense that since Dan Meyer has done nothing but criticize KA since the very beginning, this MTTK Prize is simply a way to criticize in the guise of altruism.

    This is pretty easily disproven:

    Anyone who puts the kind of expansive warranties as Khan has on a product as important as education, deserves all the fair, lucid criticism we can muster.

    This thread has expired.