Your attached article had a sentence that struck me as being at the heart of many things we do, not just math. It was that people engage in activities where “individuals have learned to participate, rather than on knowledge that they have acquired.” As a teacher, I see those times that children are performing a memorized algorithm that carries as much meaning for them as teaching my beloved puppy to roll over for a treat. My best students do not like problems that require explanation because they are good at performing, not understanding. My special goal this year is to encourage a fear-free exploration of what algebra really is, not just the rules for answering the “Chinese questions”. We will see. This parable is shockingly accurate for what I see that math has become. I am looking for ways to incorporate the term “literacy” in my math classes. The reading for understanding won’t be taking complicated texts about math and annotating them. Literacy in math is about understanding the numbers themselves and how and why they work in the beautiful and logical and intricate patterns in which we find them. So, literacy in math isn’t words, it is numbers! Have a 6 day!

This parable has so many contextual elements surrounding it, it is impossible to know what you mean when you say it is perfect. Perfectly achieving what?
Anyway, let’s use it as a chance to riff. I first encountered this parable 4 years ago reading Steven Pinker’s “How the Mind Works” (which is a stunningly rich read). There he tells us (as does Wikipedia), that the parable was originally written by John Searle to convince others that, although a computer may be able to simulate intelligence and understanding, this would remain a simulation. The computer would not actually understand.
In the book, Pinker challenges this thought experiment from a number of angles.
Firstly, the person in the room can be thought of as our brain instead of a computer. Just as it is easy for us to imagine a computer program not understanding the big-picture point of what it is doing, it is also easy to imagine our brains not knowing the big-picture point of what we are doing either. In this view, our brains simply carry out the tasks of taking in the sounds of verbal language, processing them in a myriad of ways that we don’t understand, packaging them into neurological deliverables that coast toward the action muscles in our mouth and cause us to speak a response. Adding an ironic twist, we do not understand almost anything of how our own brains do what they do. So who really understands Chinese, us or our completely mechanical brains?
Secondly, Pinker calls our attention to the fact that the speed with which the Chinese translation task is carried out may affect our perception of the translator’s understanding of Chinese. The parable works in part because, as we play it through in our heads, we imagine a person tediously and slowly looking at the characters that are input, flipping through pages of a hardbound book (how slow!) looking for the right translation rules, carefully copying the rules from book to page (probably with a quill and ink) in a step by step fashion and finally slipping the response back under the door. This slow process lubricates the path toward concluding that the person doesn’t understand. Now speed this up to the point where the person could answer people back at a fluent conversational speed or better. Does the person (or the room itself, maybe) understand Chinese now? It is not as easy to say no.
For more food for thought on this, the link below provides an example where a computer, namely IBM’s Watson, is being used to perform a cognitive task that we often associate with higher order understanding: creativity. And not just any creativity, but creativity that results in valuable human solutions to specific problems.

https://www.youtube.com/watch?v=EWF4sC5wuqs (follow the link in the video to find out more. fascinating.)

I have a deep goal to teach students to understand mathematics. But a fundamental challenge has always been to figure out what it means to understand mathematics. It is easy to read this parable and think “Ahhh, yes. Someone else understands what I mean when I say that simply becoming fluent in the rules of mathematics is not the same as understanding.” It seems to perfectly assimilate into our current views (I suspect that’s what you meant by perfect, Dan.) But Pinker’s arguments helped me turn this into a deep exercise in accommodation, challenging my beliefs. It is possible (probable, even) that a large part of understanding is the accumulation of tons and tons of specific examples of different mathematical phenomena over and over again.

I love this parable !!
People whose job is to mend televisions (old style, analogue, with tubes) do the job based on a set of rules or instructions, but are very unlikely to know how a television works. I am aware that “math is not like that”, but understanding is a lot more than understanding how standard algorithms work. It has a lot to do with understanding what all this stuff is for, and without this depth of experience the “have a go” person is not going to get very near any sort of solution. We are wasting time teaching techniques for exam purposes, but we are running the risk of wasting time insisting on students giving explanations for everything. Math education will only improve when the development of problem solving skills is seen as the justification, with the tools being introduced as and when appropriate.
I remember being the man in the room when I was a student studying some aspects of abstract algebra.

Yay, the Chinese Room on an educational blog! As some of the commenters above though, I am not sure what purpose does it serve here. In the following I wishfully think that everyone knows Chinese Room and Turing test (the idea, that if something *behaves* intelligently, we might as well consider it *being* intelligent).
In the classical setting, CR says: well, something may pass the test, yet still not *really understand* what is going on (so there is no *true thinking* and the test fails its purpose). Intuitively, this is true. But what we still lack is, ahem, understanding of what understanding really is. Ironically, this is why the T. test was designed as it was: to evade fuzzy notions, such as thinking (as in *really* thinking), it goes with “looks like thinking” instead.
What do we do, as teachers? Ultimately, we train our students to respond to challenges in some expected way. Yes, we have some idealized models of mind and such, but after all, we want to see results in student performance. Of course, I am not talking simple silly rote test questions, which only show shallow application of basic algorithms. We want to see true understanding. We do not want a CR operator. Ok. So… how does a student, who really understands the matter, respond to our challenges? Perhaps we need far better challenges etc., but we are ultimately still stuck within the frame of challenge/response interaction and evaluation. We are not allowed to open up our students’ skulls and rewire or test the brains for true understanding directly.
So where are we? Back in the CR. It is in principle (e.g. with sufficient number of sample challenges, even if they are sophisticated) possible to train the correct responses. And it is even possible to still evade proper understanding. The Chinese Room does not say much more than that we need to define well what understanding really is, and that the risk of misunderstanding while responding plausibly is always present. Our challenges simply have to be good enough so that this risk does not matter anymore.

My personal takeaway for the CR in context of AI: It is not the person who is intelligent, it is the whole system, namely the translation rules. Some sort of degree of that intelligence can be devised of the rules size. The more the rules, the less system in the language do they see, the stupider the room is (in terms of “internal” intelligence, as it performs correctly anyway).
This idea of “size of rules-system” also gives me guide to avoid the CR scenario in my classroom. I may consider the student who memorizes loads of special cases as stupider, than the one who works with a general rule. But I can hardly convince him about the shortcomings of his approach as long as they both perform equally well (and some indeed prefer to use memory over thinking). Investing the extra intellectual effort into learning some deeper rules, connections, metaphors, motivations etc., what can be seen as understanding, must be worth it. If it is not, then I can hardly blame the students to look for more efficient ways to get through. There must be a problem, which a student with understanding solves, and the other does not. What else would be the point of understanding? (Enjoying the innate beauty of the subject is important, but not everyone appreciates it)

So, yes. CR is a pretty perfect parable, it illustrates well some fundamental features of Turing’s argument. Basic philosophy of AI in general has some relevant implications for teaching, since both fields have a lot to do with “faking it”. Those who do not think it through are often misled with their intuitive conceptions of what *true* thinking, understanding etc. actually is and what it is not.