This is a thought provoking post. Thank you. Does age of the student matter? I am unfamiliar with the study you cite, but I would love to know how old those subjects were. My gut: age does not matter. I think from young children to adults, the implicit learning moments would be more engaging. But I would interested to know what the data say

“Kim concludes that “stories are interesting to the extent that they force challenging but resolvable inferences on the reader” (p. 67). So consider a design principle for your math classes or math curriculum: “Ask students to make challenging but resolvable inferences before offering them those resolutions.” Start with estimation and invention, both of which offer cognitive benefits over and above interest.

The use of estimation is quick and seems effective, so I do not disagree here. But the studies on “invention” and “productive failure” are textbook examples of poorly controlled education research. Recently, a number of studies have controlled for the weak DI conditions Kapur and Schwartz used and found that DI results in comparable or sometimes superior learning. See http://bit.ly/1RjFZkO and http://bit.ly/1Oc57nj

Here’s an excerpt from the abstract of the first study:

1. Problem-solving prior to instruction triggered a global awareness of knowledge gaps. While this was beneficial for learning when combined with instruction with student solutions, our results indicate that comparing student solutions during instruction to specify the gaps is the most relevant factor.

First, let me say that that is a deeply weird choice of “sentences” to use as an occasion for measuring learning versus interest. For one thing, I find the grammatical and punctuation errors distracting. For another, the inherent cultural biases in the “story” are also very distracting (where on Earth does a situation like this occur?). I can’t imagine any of my students not asking why in the name of everything holy we were wasting any time on that sequence of sentences at all.

Even if I don’t have a good estimation or invention starter on hand, it’s well known that I can dramatically improve the effectiveness of my lecture sequences (or snippets) and my students’ learning gains by providing them with some occasion to “prime the pump” of their own memories and internalized experiences around a given math topic.

Just yesterday I had student groups in Algebra 1 do a table reading of a “deleted scene” from a Harry Potter movie that I claimed to have “found” on the topic of “more than” and “less than.” This was our opening to a unit on inequalities.

Sometimes just giving students a chance to disinhibit themselves – while they re-expose themselves to a topic – can improve receptivity a lot. :)

For me, the larger point is that *mere* direct instruction is an impoverished form of teaching, especially if you are trying to teach for understanding. I’m a good lecturer and a respected guide to mathematics, but I have learned through experience that the only way to get kids to really digest the “expert model I am lending them” (as How People Learn puts it) over the longer term is to get them to prime themselves for learning by giving them a reason to *involve* themselves with the learning.

– Elizabeth (@cheesemonkeysf)

I may disagree with David. I think skilled writers often leave the reader to make inferences in order to generate interest (indeed often playing upon those expectations to create a surprise or a twist at the end – for example, discovering that the wife was secretly aware of her husband’s potentially fatal allergy to clams all along!).

The scenario presented was rated as more interesting when ‘step 4’ was omitted – which Dan suggests demonstrates that implicit instruction provides cognitive benefits compared to explicit instruction. However, (for what it’s worth) there are two reasons why I think Sung-Il Kim’s study and Willingham’s paper on the power of stories is not incompatible with Greg’s position on explicit instruction.

1) University undergraduates would be able to draw upon a great deal of prior experience in order to make causal inferences in the version of the story without step 4. For example, experiences of ingratitude when one has taken effort intended to help or please another person. I suspect Greg might agree that where prior knowledge is strong, then the benefits of explicit instruction start to decline.

2) The scenario is based upon what David Geary might call ‘folk psychology’ – easily acquired and rapidly learnt information about people which had benefit for our survival and reproduction in our evolutionary past.

http://evolution.binghamton.edu/evos/wp-content/uploads/2008/11/Geary01.pdf

The task undergraduates in making an inference in this scenario would be very different to making inferences in a ‘secondary biological knowledge’ domain like mathematics. In short – undergraduates making social inferences from a story is not like 12 year olds learning maths.

A good example of this is the difficulty of the Wason card test:

https://en.wikipedia.org/wiki/Wason_selection_task

Even undergraduates find this inferential logic task very difficult and the majority get it wrong. Cosmides and Tooby changed the context of the task to one involving social cheating (catching underage drinkers) and found that most people could easily make the correct logical inferences in this form.

So – I think you’re both right. Where a domain involves biologically primary knowledge or where students have strong prior knowledge or experience to help them – implicit instruction can create interest and potentially lead to a great deal of ‘focused thinking’ which will likely help them learn. On the other hand, this doesn’t undermine the argument that where the domain is abstract or where students are novices, explicit instruction may be a much more effective way to learn.

Maybe there are two kinds of implicit instruction under discussion here — micro-explicitness and macro-explicitness

OK that language is terrible and I’m sorry for jargonizing this.

What I’m trying to get at, though, is that even the “implicit” version of that passage is still pretty darn explicit. Storytelling is pretty explicit. True, leaving an inference for the reader to make leads to more interesting storytelling, in this study. But if reading this passage is a form of instruction, we’d still have to rate it as pretty explicit instruction.

I think this study tells me that, on the micro level, leaving resolvable inferences can make a form of instruction more interesting. But I don’t see what that can tell us about the macro structure of a learning experience.

Like, from this study it’s clear to me that a lecture will be more interesting if it includes little bridgeable gaps for the listener. And if you’re going to have work on a problem-solving experience, it will be more interesting if there are more resolvable inferences sprinkled across it.

But does that allow us to compare how interesting an interesting lecture is vs. an interesting problem-solving experience?

I’m caught in the middle on this.

1) I work with far too many students who are not captured and engaged by math. I reject Dan’s basic premise (as I understand it) that kids will be intuitively interested in a perplexing problem. I know far too many kids who, given choice between engaging in math and just tuning out and taking the F, will take the F. They can do seat time in summer school. Other kids simply aren’t interested in big, wide, far ranging problems. To be specific, they would not be interested in how long it takes the tank to fill up, nor would they care how many postits would cover the surface. I couldn’t engage them with open-ended problems like that, at least not in the early days of a class.

2) Likewise, I reject Greg’s basic premise that knowledge breeds competence and competence breeds interest. I read a post by Greg once in which he described a class discussion/explanation. I know he’s not talking about straight lecture. But the reality is, even a good class discussion (and I count that among my strengths) will have a good number of students just not paying attention, awaiting the moment I will come by and personally explain it to them.

So I want to increase my students’ tolerance for both. I want them to be willing to challenge open-ended problems, and I want them to tune in when I need to explain something. This has nothing to do with whether or not to do explicit instruction, but *when*.

A principal once told me that engaging unmotivated kids begins by ensuring they can do whatever task you set in front of them. I have found that to be true. And I’ve built on that discovery: increasing a student’s (often initially non-existent) confidence in his own ability to do the task in front of him also increases his trust in me, and his willingness to take on the unknown. So over time, I can set tasks that aren’t necessarily familiar at first, require him to use existing knowledge and build on it.

However, anyone who wanted to examine this process critically would unhesitatingly describe it as “dumbing it down”. But over time, I’ve found that stepping back allows me to take on more challenging math over time, with a better chance of them remembering it. I describe that process here: https://educationrealist.wordpress.com/2013/12/07/the-release-and-dumbing-it-down/

Finally, I’ve found that the more students do themselves, the more likely they are to remember the experience, when compared to my undoubtedly fabulous explanation. This does not point the way to discovery or open-ended projects, but rather tasks that allow them to use their existing math knowledge to take the next step. Then, I come in after the task to do cleanup, explaining what they found and putting it all together. They are more likely to listen because they are fully aware of the context and the math behind it. And it allows me to limit my explanations and make them more likely to tune in when they are necessary.

Way too long an explanation, but I guess I’m saying this: To me, the important thing is not whether the story is more interesting with or without the sentence. The important thing is whether or not they are going to be interested in the first place.

Ed – The feeling is mutual. Whenever you go off on a long, anecdotal ramble I tend to get a little drowsy too ;-) I will let others decide whether they prefer argument from evidence or anecdote.

I will pick up on one thing – When I hold a class discussion, pretty much everyone is paying attention because they know that they might be called upon at any time. I’ve taught in a wide range of high schools including a ‘school facing challenging circumstances’ in London and it was quite possible to have such discussions there.

Btw, maybe Greg can comment on the Kalyuga paper, as I think he is ‘close to the fire’. But I realise this might a bit off-topic.

Christian’s comment on balancing techniques like PF, PFL, and more explicit instruction deserves more discussion. I agree with him and highly recommend reading
Kalyuga’s book on instructional guidance (http://amzn.to/1mGDHzm).

I did not mean to say, in any comments above, that vicarious worked examples followed by problem solving are sufficient for instruction. I just want to raise two points that are not often brought up:

1. Many instructional guidance studies (PF and PFL are just a few examples) lack control. Unwarranted causal claims are made and that is never OK.

2. Evidence exists for multiple sequences of guidance. For example, metacognitive instruction is amazingly successful for promoting far transfer (read http://bit.ly/22OhNux for science instruction). But it should be taught explicitly and
in small doses distributed over time.

My working theory is that when information is simple (learning facts), the generation effect is optimal. When lessons increase in complexity, use a high-low sequence of guidance for the most difficult lessons. For moderately difficult lessons, use a low-high sequence (PF or PFL with scaffolds) to impart metacognitive and problem-solving skills.

It fails to imagine a student who is competent and disinterested simultaneously.

But how many of those are there? Really. I’ve met a few, but they tend to be a small minority.

In any case, they tend not to be much of a problem, precisely because they are competent. If you get a class moving along they will go with the rest, because it’s not too hard for them anyway so they might as well.

Very much more common are those that wish to be involved, are happy to be “interested”, but don’t make much progress. They are the real problem for most teachers.

If you focus on interesting them, my experience is that you don’t improve outcomes much. They enjoy Maths, sure, but shy from repeated application to build really solid understanding. However, if you can get them into the habit of doing quite a lot of work, then they start to pick up. Confidence follows, and interest often.

However we teach, some students will make more relative progress. Any style is easy pickings if we concentrate on it’s worst possible sub-group, especially in the theoretical, without evidence that it is actually a problem.

I also don’t care for “evidence” from PISA, because there’s too many cultural unknowns. I could argue that countries who move to where interest is prioritised as a starting point, before confidence via skills, see a slide in grades. Isn’t this what happened to Canada?

@ Greg Ashman

You are correct about my tentative theory. It’s a hunch.

1. Regarding the use of low guidance, I do not necessarily recommend PF alone. I do, however, think that PF tasks could be coupled with metacognitive instruction (read the JOEP paper above). Giving metacognitive prompts and problem-solving activites has excellent learning outcomes. The intervention I linked above was brief (several hours) and resulted in far transfer and increased motivation. Of course, I would always like to see more empirical evidence. It is also unclear how much metacognitive training is optimal. My guess is at most 15-20% of a school year.

3. It may be that well-designed PF works in the lab but is hard to implement at scale.

Some work on MetaCognitive Tutors may be helpful http://bit.ly/1UB3i78

This has been a nice chat. I’m currently working on a master’s thesis in this area. Let me know if you would like to discuss this further through email (vpletap@outlook.com).

Dan: I don’t think that PF works better than EI for some classes of problem. I just think it is less likely to fail for some classes of problem. I would still use EI to teach standard deviation to students who had not met this before. This is for two reasons: Firstly, I don’t see the types of EI that PF is compared with in studies such as Kapur’s as optimal forms of EI, although I do admit that this would be hard to achieve whilst also keeping the test fair (i.e. it is a genuine problem and not a deliberate attempt to influence the result). Secondly, there are other sources of evidence to support EI from outside this particular line of research such as effective teacher research and strategy instruction research.

The fact that Schwartz and Martin were able to replicate their own study with different teachers tells us something. However, it probably does not tell us as much as a replication by other researchers who may have used a different method / control.

“The science of learning” is an interesting phrase to use. Some of those who I tend to disagree with would dispute that such a thing is even possible. They would call it “positivism”. However, I do think that we have learnt a lot. This does not necessarily mean that teachers from the 1960s are all very different from teachers today. And it does not mean that we should dismiss a whole body of experimental research just because it is a bit old. We don’t do that with any other science.

Chester, you’re completely correct.

To put it more briefly:
Before talking about how to teach, don’t we need to determine what we want students to learn?

And I am not talking about the specific content standards. Even if we accept the Common Core as a perfect document (which the authors certainly never intended), we still need to make decisions about emphasis and about how we understand mastery. I think this conversation will be much more grounded if we look at specific learning outcomes and discuss the teaching strategies in that context. I suspect that there will be a surprising amount of common ground in approaches once we decide on a particular learning outcome.