Does Johnny have trouble converting decimals to fractions? The database will have recorded that – and may have recorded as well that he finds textbooks boring, adores animation and plays baseball after school. Personalized learning software can use that data to serve up a tailor-made math lesson, perhaps an animated game that uses baseball statistics to teach decimals.
One, it shouldn't cost $100 million to figure out that Johnny thinks textbooks are boring.
Two, nowhere in this scenario do we find out why Johnny struggles to convert decimals to fractions. A qualified teacher could resolve that issue in a few minutes with a conversation, a few exercises, and a follow-up assessment. The computer, meanwhile, has a red x where the row labeled "Johnny" intersects the column labeled "Converting Decimals to Fractions." It struggles to capture conceptual nuance.
Three, "adores" protests a little too much. "Adores" represents the hopes and dreams of the educational technology industry. The purveyors of math educational technology understand that Johnny hates their lecture videos, selected response questions, and behaviorist video games. They hope they can sprinkle some metadata across those experiences — ie. Johnny likes baseball; Johnny adores animation — and transform them.
But our efforts at personalization in math education have led all of our students to the same buffet line. Every station features the same horrible gruel but at its final station you can select your preferred seasoning for that gruel. Paprika, cumin, whatever, it's yours. It may be the same gruel for Johnny afterwards, but Johnny adores paprika.
Enjoyable games/activities in general are difficult to create, especially in any quantity. Learning and teaching are complicated and personal by necessity. The combination is exceptionally difficult. [..] It’s just not realistic for this to happen on any timetable or method I’ve seen proposed.
2013 Mar 11. Michael Feldstein links up this post and wires in a comprehensive "Taxonomy of Adaptive Analytics Strategies."
First of all, the sort of surface-level analysis we can get from applying machine learning techniques to the current data we have from digital education system is insufficient to do some of the most important diagnostic work that real human teachers do.
Then there are those systems where you just run machine learning algorithms against a large data set and see what pops up. This is where we see a lot of hocus pocus and promises of fabulous gains without a lot of concrete evidence. (I’m looking at you, Knewton.)
And guess what? Nobody’s been able to prove that any particular theory of learning styles is true. I think black box advocates latch onto video as an example because it’s easy to see which resources are videos. Since doing good learning analytics is hard, we often do easy learning analytics and pretend that they are good instead.