Before I get to the good, here’s the tragic, a comment from a father about a math feedback platform that I don’t want to single out by name. This problem is typical of the genre:
My daughter just tried the sine rule on a question and was asked to give the answer to one decimal place. She wrote down the correct answer and it was marked wrong. But it is correct!!! No feedback given just — it’s wrong. She is now distraught by this that all her friends and teacher will think she is stupid. I don’t understand! It’s not clear at all how to write down the answer — does it have to be over at least two lines? My daughter gets the sine rule but is very upset by this software.
My skin crawls — seriously. Math involves enough intrinsic difficulty and struggle. We don’t need our software tying extraneous weight around our students’ ankles.
Enter Classkick. Even though I’m somewhat curmudgeonly about this space, I think Classkick has loads of promise and it charms the hell out of me.
Five reasons why:
- Teachers provide the feedback. Classkick makes it faster. This is a really ideal division of labor. In the quote above we see the computer fall apart over an assessment a novice teacher could make. With Classkick, the computer organizes student work and puts it in front of teachers in a way that makes smart teacher feedback faster.
- Consequently, students can do more interesting work. When computers have to assess the math, the math is often trivialized. Rich questions involving written justifications turn into simpler questions involving multiple choice responses. Because the teacher is providing feedback in Classkick, students aren’t limited to the kind of work that is easiest for a computer to assess. (Why the demo video shows students completing multiple choice questions, then, is befuddling.)
- Written feedback templates. Butler is often cited for her finding that certain kinds of written feedback are superior to numerical feedback. While many feedback platforms only offer numerical feedback, with Classkick, teachers can give students freeform written feedback and can also set up written feedback templates for the remarks that show up most often.
- Peer feedback. I’m very curious to see how much use this feature gets in a classroom but I like the principle a lot. Students can ask questions and request help from their peers.
- A simple assignment workflow for iPads. I’m pretty okay with these computery things and yet I often get dizzy hearing people describe all the work and wires it takes to get an assignment to and from a student on an iPad. Dropbox folders and WebDAV and etc. If nothing else, Classkick seems to have a super smooth workflow that requires a single login.
Issues?
Handwriting math on a tablet is a chore. An iPad screen stretches 45 square inches. Go ahead and write all the math you can on an iPad screen — equations, diagrams, etc — then take 45 square inches of paper and do the same thing. Then compare the difference. This problem isn’t exclusive to Classkick.
Classkick doesn’t specify a business model though they, like everybody, think being free is awesome. In 2014, I hope we’re all a little more skeptical of “free” than we were before all our favorite services folded for lack of revenue.
This isn’t “instant student feedback” like their website claims. This is feedback from humans and humans don’t do “instant.” I’m great with that! Timeliness is only one important characteristic of feedback. The quality of that feedback is another far more important characteristic.
In a field crowded with programs that offer mediocre feedback instantaneously, I’m happy to see Classkick chart a course towards offering good feedback just a little faster.
2014 Sep 17. Solid reservations from Scott Farrar and some useful classroom testimony from Adrian Pumphrey.
2014 Sep 21. Jonathan Newman praises the student sharing feature.
2014 Sep 21. More positive classroom testimony, this entry from Amy Roediger.
2014 Sep 22. Mo Jebara, the founder of Mathspace, has responded to my initial note with a long comment arguing for the adaptive math software in the classroom. I have responded back.
22 Comments
Kenneth Tilton
September 15, 2014 - 6:10 pm -Classkick: “learning skyrockets with instant student feedback”
Yeah feedback! Classkick looks great.
I especially like the feature where students can help each other. There is a degradation of quality: kids provide answers instead of hints, and they also provide wrong answers, but hey, life is like that and we prolly get more energy overall from such an environment and energy is good.
Agreed, not sure how much use it will get. I have a so-called “Study Group” feature and I have the same hopes for that but I also wonder if it will catch on. My one fun thought is that it would give those of us nerds who aced Algebra a chance to be a hero to other kids for once by answering Qs. We will see. I suspect it will turn out like Stack Exchange where I never ask a question because someone else has already asked it and the answer they got was the answer I needed.
Concerns: So is Miss Kay staying by her keyboard all night while the kids do their homework? Will she be chained to her desk while kids are working in class instead of mingling with the kids?
Will they raise their hands if they make a careless mistake? Why? They do not know they made it.
If they think they know what they are doing but in fact do not, why will they raise their hands?
Text is a tricky way to communicate. Simple emails arranging lunch can get confused. “Watch your arithmetic.” may not always suffice. What if there is a subtlety Miss Kay wants to highlight? What if she wants to offer a series of leading Socratic hints so the student can truly learn from the exchange? That could take a few
Math is algorithmically tractable, why not automate it? Because software sometimes has bugs? If three million students are using maths software, how long do you think it will be before the bugs are ironed out? Do you only use software that has no bugs? if so, you do not use computers.
And how fast will the feedback be if three kids raise their hands at once? What if she has taken a small group aside to introduce a new topic, or lead a hands-on activity that makes math relevant?When will she be back to the raised hands?
For higher order activities where feedback is not needed step by step, Classkick looks great. For 25 kids doing multi-step math problems I would rather have an automaton checking things (whether kids raise their hands or not) and the teacher strolling about to talk face to face with kids. Hey, now I am the humanist.
The overarching problem here is that the teacher in this model still keeps an iron grip on the reins. Students will get feedback when the teacher has the time. They will not get feedback unless they know they need feedback. the teacher will decide that after looking at their work.
Far preferable (and even less embarrassing than raising hands even privately) is a (debugged) automatic system that actually and reliably provides instant feedback, at least for things like multi-step math problems.
Now kids can own their own learning, wrestling with mathematics leveraging other learning aids provided by the software, then as a last resort calling on the teacher’s scarce time.
One thing *everybody* forgets is that we are now talking about individualized instruction, and that places huge demands on the teacher. This is the cost of not allowing the teacher to stand on the stage delivering instruction en masse. If you have never implemented individualized instruction you may not understand the problem.
I/I is another huge win, but to make it work we cannot let it be a black hole for teacher time and energy. For Algebra at least, Classkick does not seem like it avoids that trap.
Final point: last I heard the tragic case ended up with two good outcomes: the student is now getting along fine with the software and, second, both student and parent now know that software is not perfect. They also know the publisher cares and is interested in bug reports. Not really seeing a problem here.
John Scammell
September 15, 2014 - 6:26 pm -This app shows real promise in terms of the quality of feedback that teachers and peers will be able to provide each other in real time. I hope it’s not like many apps that work beautifully on my home network, but fail to work at my school because my district doesn’t like devices talking to one another.
Kyle Pearce
September 15, 2014 - 6:30 pm -Classkick looks like a cool platform that adds the “connectedness” that often doesn’t exist in digital classrooms. At first glance, it reminds me a bit of Nearpod where you can have students send you their solutions.
The more I work with teachers learning how to integrate technology in their classrooms, the more I realize that what works for some may not necessarily work for others. I can see this product being really awesome for some teachers, but I don’t see it in my own classroom. Since all content is delivered by myself/my students on an iPad wirelessly via Apple TV, I am constantly circulating the room and have a good understanding of where students are at. It should also be noted that we are often solving tasks in a group (either full / small group) during class, so sitting down to watch my iPad screen isn’t something I’m dying to do. Is it possible for a teacher to come back to the student work or is it all “live?”
Either way, definitely an innovative tool that is worth a look for teachers who feel like they aren’t able to get around the room during class time.
Thanks for the share and insight.
Zak
September 16, 2014 - 5:46 am -Is this only for iPads? I teach Geometry in a 1 to 1 classroom, but they don’t have iPad, they have Lenovo devices. Or do you know of anything similar that I could use? Thanks
Matt Bury
September 16, 2014 - 7:51 pm -Hi Dan,
I follow your blog with interest and think you raise issues about teaching that get too little attention elsewhere. I appreciate the time and effort you put into this.
About the app, everything you’ve listed that you feel is good about it is what you can get from any decent, modern learning management system (LMS), plus a whole host of other features. Web browser and JavaScript support for math (LaTeX & MathML) is better than it’s ever been so you’re not limited to more basic math either.
With online platforms, it’s also possible to delegate giving feedback to others, although teachers should always keep an eye on what’s happening so that they can adjust subsequent classes to address specific issues (AKA just in time teaching JITT). One possibility is to delegate feedback to older and/or more capable learners: research shows that peer-teaching is of substantial benefit to the learners doing the teaching. The technique has been in use on and off since the French revolution in the Ecoles Mutuelles, the Lancasterian system, and the Madras system, and is seeing a surge in popularity in some more progressive education systems.
I guess the difficulty is to convince the school/college to invest the time, effort, resources, training, and support to get an LMS up and running. There are some excellent free and open source LMS’ around, most notably Moodle and Instructure Canvas. Both support LaTeX. Canvas: http://guides.instructure.com/m/4152/l/40320-how-do-i-use-the-math-editor and Moodle: https://docs.moodle.org/27/en/Using_TeX_Notation
Free and open source means that anyone with the know-how can set up the LMS and offer it as a service to teachers, schools, etc. or if the school has good IT support, they can set it up themselves easily and cheaply and with no licensing fees.
I hope this helps! :)
Scott
September 16, 2014 - 11:26 pm -I’m intrigued but skeptical.
Also Dan, could you tell me what words you see here http://www.captcha.net/images/recaptcha-example.gif so that we are assured you’re not assimilated into a Borg-like edtech promoting horde?
1. Binary? win for classkick. But how much faster? Worth the investment into the environment?
2. implementation issues… perhaps. Importing a “worksheet” *may* be easier than making photocopies. But are tablets or computer ubiquitous enough that we can assume students can navigate the UI **as easily as the UI of pencil and paper** ? Maybe the pros are worth the cons?
3. Stamps for the 21st century. (Hey, kids love stamps)
But is there research on how quickly a student can identify a “semi-automatic” response generated by a template vs. the same sentiment but generated by the teacher?
Delve into information theory with me for a sec: if I am a student: do the feedback words convey more than the evidence that the teacher has spent time and attention on my work?
My hypothesis is that these templates probably fall inbetween the numerical and the “certain types” in Butler’s rankings of types of feedback.
4. Seems fine. But why this over in-person communication? Not saying its not good, just who is it benefiting? Students across the room for whom you don’t want to grant permission to change seats for a bit?
5. we’ll know when the technology-based workflow problem has been solved when we cease talking about it. Why base my class on classkick? Does something done in classkick work with… google docs?
————-
What problems does this solve for the teacher? Maybe an improved feedback throughput for types of assignments geared towards the feedback enabled by the program. I certainly feel that can be a big improvement, if feedback is lacking for students. I do like the decreased reliance on paper, but I’m wary of the increased reliance on (1) electricity and (2) wireless networking.
What problems does this solve for the student? Maybe they get more feedback from the teacher, again on specific assignment types.
————-
Are physical classrooms inefficient learning environment because of communication limitations inherent to the 1:30 teacher to student ratio? Limitations that can be reduced by this software? Are there different styles of teaching that are not limited by the things that this software tries to alleviate?
Dan Meyer
September 17, 2014 - 8:07 pm -@Scott. Strong reservations. I generally feel all these same tensions w/r/t edtech adoption. Most times the balance weighs in favor of rejecting the newer technology and sticking with paper and pencil. In this instance the balance is promising though still not an easy calculation.
I’ll respond holistically here to say that I would most likely use the app outside of class. For in-class use, I don’t think students are so shy they want to send a secret hand raise to their teacher. I like the idea of students signaling the need for help from their friends but ideally they’d be grouped up for that kind of help in the first place.
I really like the idea of tuning in for office hours, though, all of us at our own homes, and seeing who needs help or what’s been done on homework to assist me in planning. I like thinking about students helping each other then.
Mo
September 17, 2014 - 10:31 pm -Hi Dan,
I’m one of the founders of Mathspace (who you may as well have mentioned by name given you linked to it) and I read your previous post and comments with great interest. I really appreciate the discussion that’s been generated around giving feedback to students and we’ve taken a lot from it.
But for your readers, I wanted to clarify a few points that you’ve brought up, because there are some key facts that have been selectively ignored and/or not mentioned.
1. This is Ian’s (referred to as the father in the post above) attempt at the Sine Rule question: https://diigo.com/045vve
* Notice that the step was not marked incorrect, but rather as a ‘typo’
* Notice the video icon available for an explanation of that error.
* Note that in the first step a teacher would also ask the student to include parentheses to make it clear the inverse sine of what expression was being looked at. Mathspace did this automatically.
2. The second typo message was a limitation of the system in accepting only one equality statement for a line of working at a time. We acknowledged that while this was Mathematical best practice it could lead to student frustration as it did for Ian’s daughter. Our team worked hard to release this fix soon after http://somup.com/c2QoiN1Qc
As you can see we’ll now break such an answer down into individual lines for the user to maintain best mathematical practice, while at the same time making sure first time users don’t get frustrated.
The end result of this exchange is Ian’s later comment (selectively not chosen for this blog post):
“My daughter has got the hang of it now! – This crisis is now officially over- it does look like a good introduction is key to the whole thing. “
For every story of first time user frustration I can give you a story of eventual success. Like the student who had never done homework in her life because she didn’t know how but is now able to follow because problems are pitched at her level and there is step-by-step feedback and worked examples as she goes. Or the student with learning difficulties who for the first time in 5 years is no longer on an individual learning plan and has joined the rest of his class due to both his improvement from using the software and the personalization it affords teachers.
Just like learning Math requires persistence and struggle, so too is learning a new interface. To dismiss the use of technology because there is a small learning curve is akin to dismissing Geometry or Algebra. This is just as much a part of learning (particularly in this digital age) as learning the Sine Rule. As Ian points out with some perseverance his daughter has picked up how to use the software and is now happily moving along and hopefully learning. Hardly a ‘tragic’ outcome.
Now that I’ve clarified that I’d like to share my thoughts on feedback. Just because a computer cannot give perfect feedback does not mean we should delegate ALL the responsibility of providing feedback to the teacher. I hear your frustration with existing solutions but you’ve taken the opposite extreme here. The funny thing is we developed Mathspace because of our frustration with the level of feedback provided by existing solutions (largely multiple choice or single input response). The problem with this type of software is it looks only for a final answer and doesn’t understand things like 5/10 is equivalent to 1/2 and is a correct step but just not simplified. This kind of feedback can be given automatically and I don’t see why we shouldn’t try and give this feedback automatically in a timely manner. Could a teacher give that feedback better? Of course she can. Does a teacher have time to sit side by side with 30 students in a classroom for every math question they attempt? Absolutely not. Mathspace is built to facilitate teacher conversations with students — collecting student input line-by-line, saving teachers time on the routine marking, and then giving teachers that information so they can have those deeper conversations asking students – “why?”
Looking at Classkick I like it for the same reasons that you do Dan. I think it’s particularly good for open-ended questions that you love to open your classes with. But I bet you I know what’s on their future product roadmap; the ability to provide SOME level of feedback automatically and instantaneously. I know this because I talk to teachers every single day and they are crying out for it. In order to do the open-ended investigation type activities you propose (and we all love) and still get through the demands of the curriculum, they need more time. So why don’t we try and automate the parts that CAN be automated and build great tools like classkick to deal with the parts that can’t be automated?
We all want the same thing here- a way to provide teachers with insight into the steps taken by a student and not just the final answer- we just take it a step further and automate that part of it that can be automated. It’s funny how you’ve singled out the only software that combines both allowing step-by-step work and easier input for students through a handwriting interface. This is a giant step in the right direction and we should keep working at making it easier to use for students and providing the right feedback at the right time for teachers.
Can you imagine a teacher trying to provide feedback on 30 hand-drawn probability trees on their iPad in classkick? This is how it is done in Mathspace http://somup.com/c2QDjP16w
Can you imagine a teacher trying to provide feedback on 30 responses for a Geometric reasoning problem letting students know where they haven’t shown enough of a proof? This is how it’s done on Mathspace http://somup.com/c2QDQ016y
I’ll let your readers decide if done well this is a good or ‘scary’ thing. All the teachers I speak to say it’s a big step up from other digital Math products. We all agree it can be a lot better still and we all agree it will never be as good as a teacher. But teachers can’t watch while every student complete 10,000 lines of Math on their way to failing Algebra. Mathspace can.
Matt Bury
September 18, 2014 - 11:47 am -Hi,
What I understand from Mo’s comment is that automated, non-human feedback systems for math are at the level they are for writing with word processors (spell-checking and basic but unreliable grammar-checking). Would you say that characterises it accurately?
Much of these conversations also follow the assumption that the only feedback learners get is from their teacher. In some (more progressive) classrooms and some online environments, learners work with each other and help to mediate their own and each others’ learning. Just the act of explaining why you think the answer is so to a classmate has a substantial effect. Common teacher practices and classroom atmospheres tend to discourage such interaction even without answers in the back of the book and automated feedback systems. We should be cautious about not inhibiting discussions of concepts between peers and I think there is a danger of over-reliance, on the part of learners, on automated feedback systems.
For a full and detailed description and thorough discussion on peer-mediated learning, see Lauren Resnick et al’s guide to Accountable Talk: http://2012-leadership-forum.iste.wikispaces.net/file/view/AT-Sourcebook.pdf
Kenneth Tilton
September 18, 2014 - 12:52 pm -@Mo: Yep, I already picked on Dan for saying MS said something was wrong when MS assiduously avoided doing so, but in hindsight I was unfair.
Note that daughter, father, and two bloggers (Dan and Fawn) all took the note about the answer *format* to be saying “Wrong!”. To them, rejection is rejection.
As for the excellent video explanation, even an astute user such as moi did not think to hover the video icon to find out what the heck it was doing there until the third time it came up.
As for the strong “tragic” and “skin crawls” language, well, I have to admit that is exactly how I feel when I see my software mislead the user (or backtrace). One unfortunate but necessary meta-lesson here is that just because something comes out of a computer does not mean it is right.
Finally, we have the same reaction to Classkick. When I was doing private tutoring of struggling students, it was rare for two steps to go by without the need for an intervention. This is not a place for Classkick.
@Matt: that was an exciting read. Accountable Talk (A/T) sounds hard to implement and worth the trouble. Hey, I almost called my app SocialAlgebra in honor of the “Study Group” message forum.
One of the acknowledged challenges for A/T is that the talk be of high quality. Nothing beats having Socrates in the classroom, but shy of that software that is wise enough to make the student figure things out for themselves will often do a better job than some teachers. Back in the nineties I even had teachers comment that they had learned from the software how better to explain certain topics.
It will be hard work getting students let alone teachers learn how not to just give kids answers or tell them outright how to fix a mistake, but if expert system software does it out of the box — well, it might be a way for the students themselves to learn how best to help others.
Matt Bury
September 18, 2014 - 1:08 pm -@Kenneth, re: “well, it might be a way for the students themselves to learn how best to help others.” — My point is that automated feedback and easy answers inhibit responsible learning.
“Anything that a child can do and should do for themselves, and that we do for them, takes away an opportunity to learn responsibly.” — Gene Bedely.
Accountable Talk is more common in the liberal arts (at all ages) and a well-worn path. It’s simply a case of getting more teachers in the natural sciences, ICT, and maths to “liberalise” their classrooms. It pays dividends from the start and encourages a more open and relaxed learning atmosphere; great for reducing teachers’ stress levels.
Matt Bury
September 19, 2014 - 7:08 am -For empirical support of peer discussion/instruction and the value of p2p interaction, this was published today: http://www.facultyfocus.com/articles/teaching-and-learning/discussion-make-difference/
Doug McKenzie
September 21, 2014 - 8:33 am -The five course meal… fork or spoon?
I think we get trapped by some false dichotomies in our discussion of math ed. There are multiple strands of mind activity involved in solving mathematically rich problems, and it seems different parts of the brain that address each one, not to mention the part that switches between them all. Students need the open attitude they practice in a 3Act, and the persistence they get from a growth mindset (Dweck/Boaler). They need to recognize the concepts in the context … I read in Moonwalking with Einstein that grandmaster chess players, when introduced to a game position 25 moves in, access their memory much more than they access the analytical part of their brains (cortex?). They don’t think about the position as much as they recognize it. Students need fluency in a variety of math tools — arithmetic skills, algebra, visual models like bars or arrays… here they also are better off when they can “recognize” fraction multiplication in the middle of a larger problem, not have that become the problem. They need to be able to problem solve, have an optimistic attitude, be expecting and willing to analyze, be resourceful, be able to use problem solving skills like collecting data and looking for patterns, drawings, working backwards… These are all separate strands in math education that I suspect need to be addressed in different ways. How often have you spent a good three days on the area of triangles with manipulatives and drawings on grids, only to have half the class forget the formula a week later? Analysis, memory… two different things, two different places in the brain. Once they have some solid thinking experience with triangles, I am going give them sprints to practice that formula, to talk directly to the memory. Think about how you learn stuff. Play piano? If you really understand music theory, can you get away without practicing scales… a lot? We need to think about what are the best methods and tools for working in each realm… I suspect that we will find that sometimes we are debating which is better, a fork or a spoon. Depends on what course you are on.
Dan Meyer
September 22, 2014 - 4:37 am -@Mo, thanks for your thoughtful response. You’ll have to pardon my not keeping my readership fully informed on the back-and-forth between you and Ian. All of those comments remain on the record for their reading. Moreover, this process where a) a student is frustrated with software, b) a blogger highlights that frustration publicly, and c) the developer fixes the frustration, while inspiring, seems fairly unrepresentative of the state of affairs. The more likely case, which I have chosen to highlight on my blog, ends with a) the student’s frustration.
I have responded to your compelling argument for adaptive math software here.
Liz
October 1, 2014 - 3:55 pm -In college, the equation editor built into Word was pretty intuitive. The one thing that was a smidge tricky was doing square roots with (x)^0.5 and keeping all the brackets straight. So a quadratic equation would be x=(-b+(b^2-4ac)^0.5)/2a and it’d come out looking like the version in a textbook. And changing up the equation was pretty easy too.
I wonder if there’s a way to make an app that lets you insert an equation into any text field on a tablet. Then the equations could be typed in quickly and the handwriting could be saved for diagrams and such.
Also, I’m sure they know about it, but just in case I’ve passed this on to two of the professors in curriculum that do a lot of work with getting future teachers to assess whether a given app actually has benefit. Sent them a link to your review here.
Even if classkick doesn’t work out, just thinking about the benefit of it’s functionality will hopefully encourage more social learning.
Matt Bury
October 1, 2014 - 4:56 pm -Hi Liz, I think what you’re looking for is a LaTeX and MathML integration like MathJax: http://www.mathjax.org/
There are already modules and integrations for some of the most widely used website software and learning management systems and it’s free and open source. They don’t list it here: http://docs.mathjax.org/en/latest/platforms/index.html but Moodle (http://moodle.org/) supports it.
Josh
October 8, 2014 - 8:11 am -Hello!
I am going to give this a try tomorrow with students. I have already have lots of those self correct quizzes and do think its great for kids to get instant feedback–but not worth the incorrectly given negative feedback.
Anyway, does classkick allow a teacher to easily save a students work?
Thanks,
-Josh