Great points here, Dan and Justin.

I’m not surprised a post like this follows your “Developing the Question” post series, since we are again too focused on the answer. Here, instant feedback is great for a procedure-based solution, but not so great at measuring any conceptual understanding.

Could they improve by having students jot down (in point form) why they perform each step in the solution, rather than only when making a correction?

I used to do a lot of “warm ups” using google forms, but do so much less often for this same reason. I’d always think “great, they get it!” but probably should have been saying “great, they can get the right answer” (but unsure if they really get it). I may start doing this again with a justification textbox for each solution to give me more insight.

Dear Dan and others –

Thanks for your thoughts on Mathspace. Feedback is something we value highly – both feedback from math teachers, as well as the feedback we give to students. It is something we think about every day, and you can see this in our philosophy: we give students feedback at every step of every question. In our opinion, this is a better pedagogical approach than other digital products which only give this feedback at the end of a question, when students have written a single right or wrong answer, or chosen a right or wrong multiple choice response. Immediate feedback is also in our opinion preferable to feedback given a week later, or whenever a teacher gets to grade the work.

Your feedback has certainly given us a lot of food for thought – for example, how should we give feedback – with a comment? Or instead, by prompting the student with a question to get them thinking? Also, we know there are times when immediate feedback is not such a good idea – so we work every day to find that balance with educators. All good things for us to work through.

However, we never intended Mathspace to be used without teachers – in fact, quite the opposite, we think it is a tool to aid teachers and students in their teaching and learning. I’d like to offer you (and anybody else who’d like one) a full demo of Mathspace so you can see how Mathspace works, and make a more informed commentary – I think you’ll be in a better position to comment on Mathspace after seeing our full product, than just based on a short GIF.

You’ll get to see that, for example, teachers can see every step a student writes, so they can, as you suggest, then go and ask the student: “why did you change your answer here?” For us, technology isn’t intended to replace the teacher, but to empower teachers by giving them access to better information to inform their teaching. My offer stands for all interested, please get in touch at dtuhoa@mathspace.co

Regards,
Daniel Tu-Hoa

@Daniel, I appreciate what you’ve shared about your goals and perspective at Mathspace.

@Sean, your comment drew out for me another problem raised by automatic feedback–namely, where mathematical authority resides within a classroom. In your description of the teacher circulating and clarifying student misconceptions (etc.), it sounds like mathematical authority resides in the teacher. They’re in charge of knowing what’s right and wrong and why, and to help students accordingly. But something that isn’t a part of the tableau you described is the authority and sense-making that happens within and especially among the students who are doing the mathematics. It’s not just the teacher who can prompt, redirect, and explain–it’s every person who’s in the room. And of course everyone’s at a different place and can contribute different things, etc.

And so this problem emerges: does Mathspace or something similar become an authority in a classroom? What kind of authority? Does its presence undermine students’ own authority, their capacity to make sense of mathematics? Dan broaches this question in his post–is the kid who erases that negative sign doing so because they’ve explained to themselves why it should be erased, or reflexively because the computer told them to?

I don’t mean to make the best the enemy of the better, and I agree that the alternatives of “either a) nothing at all or b) a short prompt when you really needed more support” are poor options. But I do think that certain kinds of feedback delivery can undermine the building up of mathematical agency and can radically determine the mathematical culture within a classroom. I hope for all students something better.

I would propose that when a student gets an incorrect answer, the software creates checkboxes next to each step and requires the student to check the step where the error occurred. Then Mathspace would provide the student a workspace to rework the problem from where the error occurred. As a teacher I would then want the student to check his work with another student. Tools with immediate feedback work best when the teacher is able to immediately review the students work. Otherwise you have a situation where students as in this example are continuing down the wrong path without understanding their error.

Further to #10 ZLBrown: Could the working be submitted a line at a time as a matter of course so that the student receives feedback all the way through, rather than just at the answer? Along with an affirmation (“Yes, you need to collect the terms by adding 8 to both sides” [framed in appropriate language]; “Yes, you need to multiply both sides by -3”) or a warning (“No: you multiplied by +3; this was not right”) this would encourage checking every line.

Admittedly laborious but perhaps good for early trials and could be switched off in later examples?

Not subtle enough to provide suggestions but would indicate immediately when the error had occurred which should mean errors don’t get ingrained. But I don’t know how tricky that level of feedback would be to implement. Once implemented it might be a boon!

The example presented here shows clearly why technology elements can’t replace humans. However, I’m a little less alarmed perhaps about this specific example. This student *does* know how to solve this particular equation… and many, many algebra students *do* know the algebra but still make careless sign mistakes. A simple intervention from a human teacher would put this student back on track in a jiffy. None of us really knows *why* the student changed the solution from -48 to 48. Maybe the immediate feedback for that very mistake was enough of a reminder for the kiddo and now he/she won’t make that error again.

I’d love to explore this software more. One example doesn’t affirm the tool or make me fear it. I’d argue that some of the (amazing) Desmos labs also affirm incorrect student thinking (students are able to create graphs over time that are not functions in “Function Carnival” for example, and the immediate feedback/animations that result are a fun reward for messing around… ha!)

Appearances can be deceiving — especially in math where the longer I teach the more amazed I am at the inventiveness of students when it comes to working out their own systems of understanding and problem solving. I’m not a fan of this kind of ‘deus ex machina’ approach to helping students do better in math, buy I don’t suppose it’s going away any time soon, so it’s critical that educators know how to choose a package that will get them more than better statistics.

This kind of oversimplified feedback in online teaching packages reinforces in students the importance of being right and moving on. It also strengthens their tendency to ‘game’ the system. I think an appropriate response to that kind of rapid error correction might be to integrate at least 2 more similar questions into the exercise on the fly so that if the original exercise had 10 questions the adapted one would have 12. The idea that quick fixes result in more questions to do might motivate students to check that their original answers make sense before submitting them. If one of those additional questions was answered incorrectly, the system could send a flag to the teacher or even freeze the exercise to further curtail students’ inclination to just keep guessing at changes and hitting ‘submit’. If, on the other hand, a student made an error which required more comprehensive correction (changes in the work as well as the answer), that might result only 1 additional question for practice.

In James Paul Gee’s incredibly insightful talk about learning in video games, he brings up the vital point that our symbol systems (written language and mathematical notation) are referents for real, tangible phenomena, and that if we haven’t had ample experience living in the worlds that the words and numbers refer to, the symbols are meaningless. He illustrates: courts have ruled that two people have the same opportunity to learn if they received the same book. He disagrees, pointing out that if one of the two people has, say, 10,000 hours of experience working with the stuff the book is about, then that person has an enormously better opportunity to learn from the book than someone who has 5 minutes of experience with the same stuff.

The same is true of feedback. The feedback that Mathspace gives has vastly different effects on different people depending on their level of experience with the material being worked on. If I put my 6 year old daughter on the 3-step equation worksheet, it really is nothing more than a frustrating, boring wild goose chase.

When I played with the 3-step equations worksheet, on the other hand, I started to wonder about how the computer was checking the answer. I started with (-7x)/5+2=37. Next I typed (-7x)/5+2=((37)), just putting two parenthesis around the thirty-seven. It told me that it was a correct step. I conjectured that, so long as I wrote an equation whose single solution was x=25, that it would tell me that I had made a correct step. So next I wrote 2x=50 and, yep, correct step. This made me perceive the interesting fact that all equations whose single solution is x=25 are equivalent and I imagined using the tool to build new, extravagant equations getting immediate feedback on whether or not they were still equivalent to the original.

So, to start, I decided to multiply both sides by x, which resulted in 2x^2=50x, and as I thought it would (because I obeyed the laws of balancing both sides, I vaguely said to myself in my head), it told me that I had taken a valid step. But then I realized that, although x=25 was still a solution, so was x=0. So now I conjectured that it checks to see if 25 is a solution, and if it is, then it verifies your step as a correct one. But more interestingly, I perceived the idea that the set of all the equations whose single solutions were 25 was a subset of a larger set of equations: those where 25 is one of the solutions. This then immediately connected to my knowledge of nested function ideas (e.g. that a line can be thought of as a parabola whose “a” value in standard form is zero). Bam! Now I’m really learning, generating new knowledge structures and assimilating them into my current ones. All because of this new type of feedback tool.

So, my daughter uses the tool and learns nothing memorable. I use the same tool and have a deep aha moment about equations being able to be organized into nested sets based on their solutions. Clearly the feedback has drastically different effects on each of our experiences as well as what each of us learn.

Of course these are two extreme examples that fall outside of the designers intention, but they help us see the range. I hope we can more readily imagine the infinitely many in-between cases such as a student who has only learned to solve one-step equations working on Mathspace’s three-step equation worksheet vs. someone who has completed the unit that includes three-step equations and is using the tool to review for a final exam.

The point: the idea of *inherent* problems with certain feedback systems is potentially misguided because of the wild variety of possible prior experiences.

@Justin: Good to see you on here. Not sure if you remember, but we took a course together at Exeter a few years back. This has implications for your authority angle as well. The way I was using the tool, I was using the feedback mechanism as a servant of my investigation. I certainly saw it as an authority, or expert, on the topic of knowing whether or not an equation has a solution of 25 (it is much better at that than we will ever be), but I was engaging with this authority as a collaborative peer. My point here is that the type of belittling student experience you are warning against, the type that makes students think that they can’t make sense of mathematics themselves, is not an inherent property of any feedback system (including Mathspace), but has more to do with the students’ beliefs in the area of mathematical autonomy before their interaction with the feedback mechanism begins. If they believe that they can’t make sense of mathematics, Mathspace may reinforce that. If they think they can make sense of it, as I do, then Mathspace reinforces that.

Harry O’Malley wrote:
“This made me perceive the interesting fact that all equations whose single solution is x=25 are equivalent and I imagined using the tool to build new, extravagant equations getting immediate feedback on whether or not they were still equivalent to the original.”

There’s a neat activity called ‘clouding the picture’ that relates to your idea in the old UK National Numeracy Strategy guidance (still available from http://www.nationalstemcentre.org.uk/elibrary/resource/4645/algebra-study-units ) I’ve used it many times and find it an effective way to introduce solving linear equations. It has the benefit of allowing massively differentiated responses too.

Where to begin?

1. Not all software providing feedback is alike. I agree much of it could be much better. Certainly providing feedback for just the answer is of little help to strugglers, especially since often the key step is the first with the rest just mopping up. I believe MathSpace and tiltontec.com are the only ones checking intermediate steps. (maybe also Cognitive Tutor?)

2. Software can evolve, with feedback from users hoping to improve it (not so much from users who just hate the genre — no useful conversation to be had there).

3. What happened to the generally acknowledged value of continuous formative assessment?

4. Feedback a day later is not feedback. Feedback is immediate.

5. Why does this blog and every comment presume the teacher and not the student is responsible for their learning? Are students patients lying in hospital beds waiting for the doctor to come around and fix them? I prefer giving kids multiple learning resources and letting them figure out how they will master Algebra. We need them active, not passive.

6. If mistakes are visible to the teacher kids become self-conscious. Math anxiety is a serious problem, let’s give them some privacy as they flail about. My software does not even record failed “missions” because I want them to try, try again. If they get frustrated, why cannot they be the ones to ask the teacher or other students for a conceptual reset?

7. Why cannot the software offer a forum where students and teachers can collaborate, one allowing embedded videos and or easily edited math?

8. Some ed tech developers were/are teachers. I suspect one can tell which by their products.

8. Benny averaged over 80% applying a cleverly concocted but wholly bizarre rule system? The story did not say much about the software, but towards the end suggests it was multiple choice. Not good.

9. The Gran Turismo driving game offers several modes of play. In arcade mode one is free to crash competitors off the road and plow on to victory. In “license” mode (with more advanced licenses required to open up certain races and buy certain cars) if one makes one mistake it’s “game over”. Moral for math software left as an exercise.

10. In the example given, note that most likely the student made a careless error. If they are in fact weak on signed arithmetic, they will continue to get quality practice despite the deficit. If they are not interested in getting better, they will not get far in “license” mode. Oops, I gave it away.

11. his has nothing to do with rote memorization. Number facts are stimulus-response and can be memorized by rote. In Algebra the problems all look different and one must see past the particulars to recognize which of dozens of valid transformations can be applied and applied usefully in which order. This recognition of key abstract features offering an opportunity to simplify (or factor) is a learned ability developed only through substantial practice.

12. A surprise for many commenters (and me when I heard it): teachers like my software because kids cannot continue without correcting mistakes. Apparently without the software struggling students happily filled a page with guesswork so they had something to “turn in”. Another reason feedback must be immediate.

13. @Harry O’Malley” Great stuff. You meant “x = -25” was correct, yes? Anyway, I tried the ((37)) variant on my software and got a yellow “warning” background on that step and the avatar tutor said “I was thinking in a different direction.” Multiplying both sides by “x” I expected my software to say it only handled linear equations but apparently I have that confession only when a new problem is being entered, so it marked it as wrong. Gotta fix that. My software does fall back on numerical methods to avoid saying “wrong” to correct work (something I dread) so I have some research to do. Thx for the test case!

Hi Everyone,

Just wanted to let you all know that I took up Daniel Tu-Hoa on his offer for a more detailed tour of Mathspace and I was really impressed with where this tool is and where they want to go with it. We discussed some areas where improvements could be made such as allowing the teacher to “toggle” a justification box for students to explain their thinking as well as a “stream/feed” that highlights areas where students were struggling. If/when those are added, that could help take care of some of the original concerns Dan and others had mentioned.

I did also mention that while Mathspace is really advanced compared to many other online learning tools, I still have concerns that this tool may not improve the motivation of students. We all know that many students who have struggled in math for some time have likely lost motivation to improve. If motivated to learn, it seems as though a student could learn as much as they desire.

The real question here could be: How can Mathspace be improved to bring out that curiosity that Dan and so many others continue to search for in effective math tasks?

If students are being assigned Mathspace problems, but are still doing the work “to get it done,” will they be any further ahead?

I understand what you’re saying 100%. The immediate feedback can keep our kids from understanding.

BUT…….The way they turn writing into mathematical equations!!!! This is awesome! And they just started – no need for the Khan-like hate mail yet. Even if nothing comes out of their startup except the licensing of whatever magic they created that turns my ridiculous scribbles into beautiful equations, let’s give credit to the genius that created this.

This could help launch a lot of other really interesting (and non-immediate feedback) apps for our students!

“When you make a mistake in math, your brain grows. Synapses fire in your brain. In fact, your brain grows when you make a mistake, but when you get work right, no brain growth happens…”

…says Jo Boaler here:
https://www.youtube.com/watch?v=qD5QR5R6b8E

Dylan Wiliam also referenced this phenomenon in a recent workshop I attended, calling it the “hyper-correction effect.”

All this to say, technology now, more than ever, can equip teachers and their students to receive instant feedback through formative assessment tools. Making mistakes is part of that process. Knowing about those mistakes instantly and having the opportunity to correct them and/or receive an intervention from the teacher would seem to aid the sort of brain growth Boaler and Wiliam are talking about.

No tool is perfect, no teacher is perfect, and no student is perfect. However, using tools that interrupt the learning process when students start to head the wrong direction has been very empowering for me, and more importantly, for my students. They don’t know what they don’t know, and formative tools can really help in a right-on-time way.

I disagree with the idea that if mistakes are visible to the teacher kids become self conscious. While that may be true in too many classes that is a teacher mistake and not a general rule.

Regarding visibility of errors to teachers, maybe it was just me, but I always freaked out if the teacher stopped while wandering around the class and looked over my shoulder to watch me work. I seem to recall kids basically refusing to continue working until I moved on when I did the same as a teacher.

I think if a teacher says to a kid “I see it took you ten tries to figure out 2/3 divided by 3/7” it will have a chilling effect on their willingness to take risks.

Somewhat related: I am reminded of what happened when Cognitive Tutor took points off if a student asked for a hint while solving a problem: kids stopped asking for hints and instead asked the teacher. Doh!

So Dan is right: the designs of these tools must be all over the human engineering aspect or a train wreck is possible.

As for right/wrong not being useful, two things: when contrasted with a silent piece of paper starting back at the kid it seems useful. As for something more specific than “wrong”, I have considered diagnosing the student’s likely mistake and saying, “Not quite. You might want to rechecked your signed multiplication.” But then I think we are giving too much help: kids just sail through their work fixing mistakes as prompted and never go through the pain and suffering of figuring out what they did wrong.

This, btw, is where I deviate from the adaptive crowd. Cognitive Tutor (last I looked) provided the answer after the student failed on three tries. I guess the idea is to take them back to easier problems for a while to rebuild their momentum instead of leaving them flailing helplessly at something for which they are not ready.

I like to see them flail, albeit with several forms of help available besides trial and error, including a last resort internal forum where kids can ask other humans for help when they are home alone.

Anyway, Dan’s original concern was for us devs to make sure we are making new mistakes. Not to worry, we are.

Harry was definitely right, these comment boxes should be catching mistakes such as “Mathspace has told her she’s wrong.” Actually, it was quite careful to say “Uh-oh…you must enter an equation.”

The reason it does that is because both sides of this debate (but not all software developers) worry quite a bit about giving false information. I dread it, myself, and have quite a bit of code focused only on that.

For example, before saying something is wrong based on my own symbolic verification I use numerical methods to see if maybe the kid is plausibly right. If the numerical approach supports their work, I do not say it is wrong. (I forget if I still “say, hmm, you may know more than I do. Carry on”).

In this case, where they just type “3”, much like Mathspace I say “Typo? I ask because the problem statement involved an equation and this does not.” (I do not offer a Q&A format like Mathspace’s, so I just entered “n+7=10” and then “3”.)

I was surprised to see my app marked “3+7=10” wrong, but then remembered it supports problems in which students are asked to identify whether an equation is an identity, contradiction, or a conditional expression (and this is a conditional equation with solution n=3.

Now if this were a system-generated practice problem Fawn could have asked to see it solved, and discovered the expected format. But I had entered the problem myself. I was able then to ask to see a similar problem solved and it came up with “n-6=7” (the random selection of the same variable will completely confuse kids, I will fix that) which it solved via “n-6+6=7+6 and then “n=13”.

So what do I have to do to piss off you Feedback Haters badly enough to QA my site for free?! Mathspace will indeed get from this some ideas for improvement, such as saying instead “You must enter and equation some in the form of n= “.

Meanwhile: the FH crowd has no problem with kids working 10-15 problems without feedback? Have you ever tutored one-on-one? Did you correct them as soon as they made a mistake?

Condescending? You mean the questions about how you all want practice handled? Sorry, I honestly meant to raise the question of whether you all think substantial student practice is still important, because I think for us to understand each other that is necessary.

Perhaps the answer is that a lot of practice solving problems is not desired. I certainly hear a lot of denunciation of “rote learning” and “drill 7 kill”. But if it turns out we agree that substantial practice is necessary (I said “if”) then the next question would be, from where and when should the feedback come? Perhaps the answer is traditional: in a group setting where everyone goes over the work. I taught successfully that way, and I hear a lot of discussion nowadays about kids explaining rationales so maybe that is what is desired.

Otherwise, I should think the conversation should be about how to help us improve automatic feedback.

So really, I was just trying to move the debate out of what seemed to be a deadly embrace. Perhaps we are so far apart I should just ignore this blog.

If you meant my general tone, yeah, I am a piece of work. But given your characterization of serious systems produced by serious math educators as “these terrible systems” and “lousy feedback” and Fawn’s suggestion that we were “shitting” her I thought it was OK to join in on the fun.

ps. I was joking about the feedback. We are not yet even in beta and the first feedback I will seek will be from those who respect what we have done and want to fix the new mistakes we are making. Peace.

Ah, yes, Aplusix. Thx. I had found that then forgot the name and my google fu failed me trying to find it again. I had remembered it as the only other sight that had a nice maths editor. (now there are a few.)

As for intelligently reducing the help, I cannot argue: as a tutor I used to downright pretend further factorization was possible if they glanced up at me tentatively to see if they were done.

As for feedback being a problem, yep: I remember the client who instantly corrected every mistake as soon as I raised an eyebrow, but then made another mistake straightaway. I pointed to twenty problems and said you have $5 from my fee and lose one for every problem you get wrong. The cash on the table set him on fire. He would do a problem and then put his hand in my face and say, wait, I am still checking. Where he would have gotten 18 wrong based on his rate error he ended up making $2 — and I never saw the slapdash behavior again.

DME sounds interesting with the gradual withdrawal of help, but I prefer Gran Turismo: it lets me decide how to learn.

I think adaptive (where the system decides how I will learn) is a mistake. I got schooled one day watching a student use my software. She had set the difficulty level at easy and was doing problem after problem after problem correctly. I said, hey, go for medium. She shut me down: “I want to do a few more until I am ready to move up.” ha-ha, right, learner differences.

Too much of what I read on ed blogs these days is about teachers having the information they need to micromanage to perfection the learner experience. The image springs to mind of Nurse Ratched behind the bullet proof glass dispensing medication to the unwilling patients. “OMG, if i do not know what they are thinking they are doomed!!”

Far better would be to get the student engaged in figuring out how best to master a skill. Just having them aware they are the active and responsible agent is a huge win. Second, hey, they know how they like to learn.

Me, i cannot for the life of me read a computer library specification. It would be great if I could, but I cannot. Now get out of my way, I am going to write some code in this language i do not know. I will read JIT (just in time) when I have a reason to read.

the approach I and Gran Turismo take is a lot more trusting of the student: (1) here is an unbending standard that will adjust neither to your issues nor your parents’ threats; (2) here are four different kinds of help you can have to master the material; and (3) knock yourself out.

The Benny horror is simple: too low a threshold, too much help during the test. I would blame the teacher for not following the program and spending time circulating amongst the kids, but teachers, too, are human: the free time afforded to them by automated feedback disappeared in the time-energy deficit every teacher, um, enjoys.

-kt

My daughter just tried the sine rule on a question and was asked to give the answer to one decimal place. She wrote down the correct answer (formatted properly using the editor):

x = 19/sin(91)*sin(36) = 11.2

and it was marked wrong. But it is correct!!!

No feedback given just – it’s wrong. She is now distraught by this that all her friends and teacher will think she is stupid.

I don’t understand! It’s not clear at all how to write down the answer – does it have to be over at least two lines?

My daughter gets the sine rule but is very upset by this software.

@Ian: Can you tell me what section that is in? I’d like to dive in and see what I can see.

btw, I do have the free MS trial and on a different problem after I made a mistake the “Hint” button led to the next step in the solution. You might be able to figure out what is going on that way.

To avoid this sort of thing (not even knowing why the software says “wrong”) on missions (no second chances) my app marks the work and shows the solution.

On practice, you can ask to see the problem solved and explained in steps.

Only on student-entered homework do I not show solutions.

As for the answere being marked wrong, I am sure MS dreads doing that as much as I, for exactly this reason.

I’ll go buy a year license now and see what I can see, but a pointer on getting to that kind of problem might help.

ps. No way those sin arguments are meant to be in radians, is there? 91 sure does not look like radians.

I’m not sure of the question number, but it was in the sine rule section.

I just tried myself with the free version. I have to be honest it is completely painful and awful to use! All the time and effort is spent on trying to input the expressions, and it is really unintuitive.

It took me ages to do a simple sine rule calculation. One needs to start using the weird formatting which is so odd.

I think I found the problem which prompted my query.
I was told I got another question wrong, when indeed it was correct! I had two equals signs on the same line which was marked as wrong. When I put the same answer on a separate line, it was marked correct.

So, to give a simple example:

I think:

x = sin(30) = 0.5

would be marked wrong!

But

x = sin(30)

x = 0.5

would be marked correct.

Obviously both are OK, but a confident person would just do the first answer and be mystified why it was marked wrong.

I am just starting to explore this site http://www.aboutus.assistments.org/learning-increase-with-homework.php but check out what the kid says at 2:04 in the control group (no feedback) video:

“I think I did all the division problems wrong.”

Precisely. That said, full marks to the naysayers: there is a lot of bad software out there, keep holding our developer feet to the fire.

Thanks for the helpful comments.

My daughter has got the hang of it now! Phew – what a relief, both to her and the parents! She writes the derivation down on a piece of paper first, then is careful to input it in the computer line by line, and finally gives the answer to the required number of decimal places. I’m not sure what happens if you provide more decimal places than required. Hopefully it lets you carry on and write another line with the correct number of spaces. She doesn’t want to risk trying too many decimal places as she now understands what the program is looking for, and is still scared still of typing it in wrongly despite having clearly understood everything.

I’m a bit old fashioned, but I think you should never be afraid to make a mistake especially when learning things for the first time. Indeed making a mistake is really useful for learning how to do something. I don’t like to see my daughter petrified to input the wrong thing, even though she seems quite capable of doing all the questions.

I guess the software would be useful to students once they have gained confidence in using the editor. But displaying the scores so that her friends can also see, makes it all seem like very public summative assessment which understandably will scare the poor kids too much!

Another thing I don’t like about the questions with angles, lengths etc is the appallingly inaccurate diagrams! A recent question had an angle of 6 degrees marked which looked more like 60 degrees! I’ve noticed this in other software, books etc that she occasionally asks me about. Also, some questions are impossible unless you assume a particular angle is a right angle or two side lengths are equal, and yet there is no indication on the diagram or the question wording that is the case.

It doesn’t take much effort for software to draw an accurate triangle with the angles of the right size (or reasonably close). I know concepts are the important thing, but when questions have rather weird angles in integer degrees and it’s quite easy to draw the diagrams properly, then it strikes me as just being lazy to draw a geometrically dissimilar triangle – and potentially confusing for the students.

@Francisco Ah yes, good point. Perhaps grossly inaccurate pictures should be avoided if possible, but I see the problem. Perhaps that is another reason why the angles seem to be rather precise – 6 degrees, 43 degrees etc. I think the angles were rounded to 10 or 5 when I learned the stuff – a long time ago!

@Kenneth

>I was/am a hypersensitive learner myself so I feel for
>your daughter. I am curious to see if she makes a
>transition to where she becomes comfortable with failure, >simply because there are no consequences. Well, if MS >works that way.

Thanks. In terms of how MS works – it may depend how MS is set up for the students by the teacher. I have no idea, and as usual now she has the hang of Mathspace I don’t get to see anything now! I’m only called in at times of complete disaster in her mind (even though it’s usually a minor problem in reality).

>Does MS have some public scorecard showing the student >rank?

It sounds like it might do, but it’s not entirely clear. She does talk about the `scores’ of other students, but it seems more like a random selection of the others rather than a leaderboard. I’ll have to try to check if she’ll let me! I don’t think it is clear to her what exactly the scores are, which may not be a bad thing.

@Kenneth Thanks! Yes – this crisis is now officially over!

You are right – it does look like a good introduction is key to the whole thing and, like your guy needed on the phone, the more sophisticated, human-like the advice, the better.

Mr. Tu-Hoa,

Six day turnaround? Is that the best you can do?

(Uh-oh. I have some competition.)

Seriously, that is wonderful work. Six days proves you have a great team and a great code base. I already knew you were dedicated to quality math instruction.

I *seriously* have some competition.

If an educator wants to know what it is like developing hard software that worries a lot about the user, I decided yesterday to chronicle just one of the bug reports I got from QA: http://stuckonalgebra.blogspot.com/2014/09/bug-story.html

Still recovering. :)

-kt

@IanD: Wait! We’re still learning!

I am dying to know if your daughter still first does everything on paper (I am guessing double-checking her work) and *then* enters it into MS.

Here is why I am so curious: my app differs from MS in that I leave (almost*) no trace of student errors. My reasoning is that I want kids to feel completely free to err.

Freedom to err does two things. First, it should reduce math anxiety. I waffle with “should” because I thought your daughter’s experience showed the opposite, but now I recall that MS keeps a record of mistakes. So maybe it confirms my hunch. Second, and somewhat related, some of us like to learn by trial and error. (You should see me coding.) Would I still have trial-and-error in my learner’s toolkit if all my thrashing is recorded? I might not have math anxiety when I started, but I might develop it!

In my corporate work I have seen users take two months to trust the software after a serious screwup caused it to start delivering seriously bad answers. I was the director of development but patiently fielded double-checks from agents until they themselves believed things were fixed.

So I am curious if your daughter decides to give MS a second chance (at direct entry of solutions) or if she is just not going to go there again.

* (I just thought of one — when they pass a mission the medal they get reflects whether they got a perfect score or got one or two wrong (you know, gold/silver/bronze). But no one can see their mistakes.