Month: February 2016

Total 6 Posts

When Delayed Feedback Is Superior To Immediate Feedback

Craig Roberts, writing in EdSurge:

Beginning in the 1960s psychologists began to find that delaying feedback could improve learning. An early lab experiment involved 3rd graders performing a task we can all remember doing: memorizing state capitols. The students were shown a state, and two possible capitols. One group was given feedback immediately after answering; the other group after a 10 second delay. When all students were tested a week later, those who received delayed feedback had the highest scores.

Will Thalheimer has a useful review of the literature, beginning on page 14. One might object that whether immediate or delayed feedback is more effective turns on the goals of the study and the design of the experiment.

To which I’d respond, yes, exactly!

Feedback is complicated, but to hear 99% of edtech companies talk, it’s simple. To them, the virtues of immediate feedback are received wisdom. The more immediate the better! Make the feedback immediater!

Dan’s Corollary to Begle’s Second Law applies. If someone says it’s simple, they’re selling you something.

Ed Begle’s First And Second Laws Of Mathematics Education

Ed Begle:

  1. The validity of an idea about mathematics education and the plausibility of that idea are uncorrelated.
  2. Mathematics education is much more complicated than you expected even though you expected it to be more complicated than you expected.

Begle coined those two laws in the latter half of the School Mathematics Study Group, a multi-decade project to figure this mathematics education thing out. I’ve heard those laws before but I hadn’t tracked down the original source until today. He seems weary in the speech. His list of tried-and-failed innovations is lengthy and disturbingly current.

Over forty years after Begle’s work with SMSG ended, those laws still offer us lots of comfort and at least a little humility. Math education is hard. My gut is probably wrong. Anybody who says differently is selling something.


Begle, E.G. Research and evaluation in mathematics education. In School Mathematics Study Group, Report on a conference on responsibilities for school mathematics in the 70’s. Stanford, CA: SMSG, 1971.

2016 Feb 26. Bowen Kerins’ links to a better copy of the entire proceedings. That site also contains links to some of the SMSG “New Math” curriculum, which I’m excited to investigate.

2016 Feb 28. Raymond Johnson cautions us not to read Begle too pessimistically:

I really do love the history of my subject and posts like Dan’s send me into hours of searching through old papers and citations. But, I must be mindful of our tendency to underestimate change when we read from our wisest predecessors. It’s too easy for us to throw our hands up and say things like, “Dewey knew it all along!” or “We’re stuck in the same damned place we were 25/50/100 years ago.” Is Begle’s 2nd law (“Mathematics education is much more complicated than you expected even though you expected it to be more complicated than you expected”) still true? I would agree it is. But, as a field, we’ve made enormous progress since Begle gave this talk in 1971. The danger, as individuals, is to not learn from this progress. To avoid reaching the same conclusions as Begle, we need to avoid starting in the same place as Begle. When I browse the pages of Begle’s final book, Critical Variables in Mathematics Education: Findings From a Survey of the Empirical Literature, I’m struck by the sheer number of things Begle and the field knew little or nothing about compared to what we know now. Don’t we owe it to ourselves, as individuals and as a field, to push past prior conclusions by starting farther ahead and taking more seriously work already done?

How Do You Make A MTBOS?

I don’t have any answers here. I can only do my best to articulate the question.

The collection of tweeting and blogging math teachers we call the Math Twitter Blogosphere confuses me.

Look at this place.

  • It has a welcoming committee that organizes challenges to help new members find their feet. It also pairs volunteer newcomers with volunteer mentors. (Shout out to my mentee Lisa Garcia.)
  • It comprises thousands of blogs and Twitter accounts. Two weekly gazettes exist just to summarize the activity. (That first link is, without exception, the most valuable post of whatever week it’s published.)
  • It organizes weekly webinars with speakers and topics running across every spectrum.
  • It organizes an annual in-person conference, which sold out in two weeks last year and in eight hours this year.
  • 2016 Feb 12. It has maintained a physical booth presence at three of the last four National Council of Teachers of Mathematics’ conferences, staffed round-the-clock by volunteers.

It bears saying again: these are all volunteer efforts and self-organized.

Someone has to help me. Does the same organization and activity exist in other content areas?

If not, then why not? Each of the efforts above boasts some talented contributors — shout outs to Lisa Henry, Julie Reulbach, Sam Shah, Raymond Johnson, Tina Cardone, and the communities they lead — but I find it hard to believe similarly talented people don’t exist in other content areas. If you had to go back in time and bet that one group of teacher bloggers would break out in these amazing spasms of collaboration, admit that math teachers wouldn’t have been your first or second guess.

So how did this happen?

I don’t get it. I love it but I don’t get it.

2016 Feb 11. Helpful data? Googling “[x] teaching blog,” I find in millions of results:


  • Math: 33.9.
  • English: 115
  • Science: 129
  • History: 73.1
  • Art: 88.9
  • Language: 91.3
  • Social Studies: 53.8

Quality, not quantity.

Featured Tweets

Featured Comments


I think of all the disciplines, math teachers are frustrated most with status quo. Personally, I want to change my practice, but on my own have struggled figuring out how to teach differently than I was taught. I don’t have a blog of my own (yet) but have grown so much by blogs I’ve found via Pinterest, Twitter, and google searches.

Brett Parker:

I think that curriculum is a huge part. Almost all secondary math curricula include the same topics. There is much greater variation in what states and districts require for social studies or even science. Different required texts for English classes.

Nathan Kraft:

In my own district, I found it hard to find anyone with much of a passion for trying new things…those who truly wish to invest the time for self-improvement. I sometimes wonder if some of my math colleagues even really like math. I certainly don’t get that vibe from the science teachers or the English teachers. Perhaps that is why I was drawn to this community. It is out of necessity in order to find those who are equally passionate.

Henri Picciotto:

But after a very few years, the group collapsed. Traffic stopped in the edWeb community, and the in-person meeting which had peaked at about 20+ people gradually shrank.

I think the fact that edWeb was a closed community, with formal membership was a fatal mistake. MTBoS is totally open.

When Will I Ever Use This?

Click through to read John Mason response to the age-old question.

I have used this response, or ones like it, for many years with teachers when studying mathematics courses at the Open University, and I have noticed that it is only when I feel I am lost, when I lose confidence, when I feel as though I have reached my limits, that I find myself asking “why am I doing this?”


They don’t actually want to know. They’re tired of feeling stupid and small.

Featured Comments

Mr. C:

Why do we need to know this? 2 words: Robot Apocalypse!

Who will reprogram the machines? Whose calculations would you trust your life with? I was sent her from the future… to be your math teacher!

Matt E:

I still love Sam Otten’s exploration of this family of questions:

Featured Tweets

Easier said than done:

I asked my Twitter team to come up with an application of imaginary numbers to dolphins. My Twitter team did not disappoint:

The Future Of Handwriting Recognition & Adaptive Feedback In Math Education

In math education, the fields of handwriting recognition and adaptive feedback are stuck. Maybe they’re stuck because the technological problems they’re trying to solve are really, really hard. Or maybe they’re stuck because they need some crank with a blog to offer a positive vision for their future.

I can’t help with the technology. I can offer my favorite version of that future, though. Here is a picture of the present and the future of handwriting recognition and adaptive feedback, along with some explanation.

In the future, the computer will recognize my handwriting.


Here I am trying hopelessly to get the computer to understand that I’m trying to write 24. This is low-hanging fruit. No one needs me to tell them that a system that recognizes my handwriting more often is better than a system that doesn’t.

But I don’t worry about a piece of paper recognizing my handwriting. If I’m worried about the computer recognizing my handwriting, that worry goes in the cost column.

In the future, I won’t have to learn to speak computer while I’m learning to speak math.


In this instance, I’m learning to express myself mathematically — hard enough for a novice! — but I also have to learn to express myself in ways that the computer will understand. Even when the computer recognizes my numbers and letters, it doesn’t recognize the way I have arranged them.

Any middle school math teacher would recognize my syntax here. I’ll wager most would sob gratefully for my aligned operations. (Or that I bothered to show operations at all.) If the computer is confused by that syntax, that confusion goes in the cost column.

In the future, I’ll have the space to finish a complete mathematical thought.


Here I am trying to finish a mathematical thought. I’m successful, but only barely. That same mathematical thought requires only a fraction of the space on a piece of paper that it requires on a tablet, where I always feel like I’m trying to write with a bratwurst. That difference in space goes in the cost column.

That’s a lot in the cost column, but lots of people eagerly accept those costs in other fields. Computer programmers, for example, eagerly learn to speak unnatural languages in unusual writing environments. They do that because the costs are dwarfed by the benefits.

What is the benefit here?

Proponents of these handwriting recognition systems often claim their benefit is feedback — the two-sigma improvement of a one-on-one human tutor at a fraction of the cost. But let’s look at the feedback they offer us and, just as we did for handwriting recognition, write a to-do list for the future.

In the future, I’ll have the time to finish a complete mathematical thought.

If you watch the video, you’ll notice the computer interrupts my thought process incessantly. If I pause to consider the expression I’m writing for more than a couple of seconds, the computer tries to convert it into mathematical notation. If it misconverts my handwriting, my mathematical train of thought derails and I’m thinking about notation instead.

Then I have to check every mathematical thought before I can write the next one. The computer tells me if that step is mathematically correct or not.

It offers too much feedback too quickly. A competent human tutor doesn’t do this. That tutor will interject if the student is catastrophically stuck or if the student is moving quickly on a long path in the wrong direction. Otherwise, the tutor will let the student work. Even if the student has made an error. That’s because a) the tutor gains more insight into the nature of the error as it propagates through the problem, and b) the student may realize the error on her own, which is great for her sense of agency and metacognition.

No ever got fired in edtech for promising immediate feedback, but in the future we’ll promise timely feedback instead.

In the future, computers will give me useful feedback on my work.

I have made a very common error in my application of the distributive property here.


A competent human tutor would correct the error after the student finished her work, let her revise that work, and then help her learn the more efficient method of dividing by four first.

But the computer was never programmed to anticipate that anyone would use the distributive property, so its feedback only confuses me. It tells me, “Start over and go down an entirely different route.”

The computer’s feedback logic is brittle and inflexible, which teaches me the untruth that math is brittle and inflexible.

In the future, computers will do all of this for math that matters.

I’ve tried to demonstrate that we’re a long way from the computer tutors our students need, even when they’re solving equations, a highly structured skill that should be very friendly to computer tutoring. Some of the most interesting problems in K-12 mathematics are far less structured. Computers will need to help our students there also, just as their human tutors already do.

We want to believe our handwriting recognition and adaptive feedback systems result in something close to a competent human tutor. But competent tutors place little extraneous burden on a student’s mathematical thinking. They’re patient, insightful, and their help is timely. Next to a competent human tutor, our current computer tutors seem stuttering, imposing, and a little confused. But that’s the present, and the future is bright.

Need A Job?

I work for Desmos where we’re solving some of the biggest problems in math edtech. Teachers and students love us and we’re hiring. Come work with us!