The code only generates data sets that are approximately normal. I would include some examples of skewed distributions. (Better yet, I would omit the topic entirely. What’s so important about box-and-whisker plots?)

I agree with Dave R and danny. I grew up in Wisconsin, so drawing box-and-whisker plots is one of those oddball California standards whose importance escapes me. Histograms, pie charts–I mean, circle graphs–and box-and-whisker plots should be left to the computer to draw. (Or have the kids write the program to do it.) CCSS says merely that students have to “display” these graphs, so I hope we can start interpreting that as using technology and working with data sets of more than 15 points.

This code will always generate 15 data points, and these points will not have any outliers (outside 1.5 * (Q3 – Q1)), so students can just pattern match and drag the lines to the 1st, 4th, 8th, 12th, and 15th places once they’ve sorted the data. It’s kind of fun the first time.

The only thing I can add (which has not already been stated) is that there is no option to include the median when finding the lower and upper quartiles. If you do this, Khan tells you that you’re wrong. (Although, I’m not sure there is a right way to do this. Always seemed to be a matter of personal preference.)

I think the largest error is the reliance on random numbers to provide a set of assessments that test an actual set of knowledge.

Random number generators are great for creating a large set of problems that are all basically the same, but in my experience you can provide better assessments/examples with a much smaller set of questions that are designed to illustrate the concept.

For example, in the dataset above where are the outliers, gaussian distributions are normal, where would the student see examples of non-normal distributions? What about data sets containing an even number of data points?

We’ve experimented with randomly generated problems but you have a hard time guaranteeing that each assessment contains the breadth of knowledge and difficulty to ascertain mastery level.

Grrr.

After a night’s sleep on this, I have a larger bone to pick:

The way we teach statistics at the middle school level, in general, is all messed up. We teach one specific algorithm for finding quartiles, when in fact it is a fuzzy measure, with a variety of methods of determining it. For crappy data sets, the different methods may yield wildly different answers. For well distributed data sets, those differences become moot.

We spend all of our time on the mechanics of one of those methods (and even then, it’s not the one that gives proper hinges) rather than discussions of when data sets are appropriate or not.

At least it’s not as messed up as our teaching of mode.

Others have danced around it, but the fundamental flaw (as in some, but not all, Khan exercises) is that you get

THE SAME QUESTION

seven straight times, without any change in structure or difficulty, even though the underlying task has a huge variation in structure and difficulty.

Also, pedagogically – if the first thing I decide to plot is the median, I’m out of luck – I have to move the endpoints and the quartiles first, to get them out of the way.

But that’s not a problem in the highlighted code.

Aside, to Brian: I agree that a single box-and-whisker plot is rarely helpful. But when you use several to compare a data set across several populations (boys vs. girls, students by grade, etc.) they are a marvelous way of very quickly seeing differences between the populations. Unfortunately, until they’re in an official Statistics class, most students don’t get to compare boxplots to each other, so their impression is that it’s just another dumb have-to-do-this-for-math-class task.

The more I am finding my way into this online math educator community the more I am loving it! The critiques of the Khan stuff and textbook activity is REALLY valuable, both as a way to show what can and should be improved by teachers (and publishers and online education organizations) and as a way of showing what these things can do.

I think Brian has a good point about finding a simple, relatively painless way to fulfill some aspects of the curriculum that are not required or set up to be in depth “good” math. I could be wrong here (and there is no need to take an entirely charitable view of these tasks or shallow curriculum requirements) but looking at it as something that should be introduced to students a year or two before looking at it in more depth is a way of doing that spiraling, non-linear education thing. It’s also a way of putting different kinds and contexts of math in view of students. Good teaching would be to provide students with some opportunity to explore these concepts more if they are excited about them and to make sure they know this is the shallow end of a large pool attached to an ocean.

I don’t know about the US curriculum or the CC standards (apologies for my new teacher ignorance; I’m only a year out of my B.Ed and an in Ontario and am trying to get myself totally up to speed on some of the HS requirements prior to upgrading my certificate this fall), but where I am teachers have a fair bit of flexibility in what is assessed for grades. All the curriculum requirements have to be assessed somehow, but not all are graded. I’m under the impression that it’s a bad idea to mix quantitative and qualitative feedback too.

Perhaps for a teacher, especially at a younger level, some of these topics would present a good opportunity to show students that sites like Khan exist (so they can learn to use them if and when they need/want/have to) and have them do one of these activities with some kind of assessment not only of how they did it but also of how they liked learning something through a website, whether they thought it was effective, etc. I guess what I’m saying is maybe we can turn some possible “flaws” in both the curriculum and free online education stuff into an opportunity to have students give feedback to us! This kind of thing isn’t as possible if we need to get students to a test ready mastery of a deep concept or help them create a project to show how well they can apply math concepts.

To push back on the question in the title of the post a bit, there’s not much we can learn about learning from how KA codes its exercise. They do such a crappy, ham-fisted, simplistic job, about all we can learn is how to colossally waste students’ time.

Hi, Ben from Khan Academy here.

Dan, thanks for pointing out that this exercise wasn’t what it could be — your feedback here is valuable.

I’ve updated the exercise so that it now includes anywhere from 8 to 15 points, so students are forced to deal with two middle numbers, both in finding the median and in finding the quartiles.

Michael, thanks for your feedback about the user interface. I’ve improved it so that when dragging a point, any other points that are “in the way” will shift over such that it isn’t necessary to drag the points in any particular order.

Both changes are now visible when you visit the exercise:

https://www.khanacademy.org/exercise/creating_box_and_whisker_plots

Brian’s comment about the Common Core standards mentioning only the construction of box plots is astute — if we go a step further and look at the 6-8 Statistics and Probability CC progression doc (available at http://commoncoretools.me/wp-content/uploads/2011/12/ccss_progression_sp_68_2011_12_26_bis.pdf) then we find that the authors of the standards in fact also recommend that in 6th grade, students are able to compare two distributions by using box plots. I agree that comparing box plots is much more valuable, so in the future, we’ll likely add an exercise for comparing plots.

We haven’t included outliers in the exercise because there exists a variety of different ways to draw box plots. Indeed, the progressions doc linked above says, “Because of the different methods for computing quartiles and other different conventions, there are different kinds of box plots in use. Box plots created from the five-number summary do not show points detached from the remainder of the diagram.” With this exercise, we’ve intentionally chosen to provide random numbers generated from a normal distribution so that students can focus on the important parts of a box plot and don’t need to worry about the differing ways to plot outliers.

In the last few months, we’ve hired many seasoned content creators to work with us on covering all of the the Common Core standards accurately. So far, they’ve created thousands of new handwritten questions that focus more on conceptual understanding to complement the machine-generated exercises we already have.

https://www.khanacademy.org/about/our-content-specialists

Finally, I just want to mention again that our exercise framework is open-source and we’re open to contributions; please submit a pull request at https://github.com/Khan/khan-exercises if you find more possible improvements to our exercises — I’ll be happy to take a look at your change.

Ben

It’s great to see Khan Academy’s responsiveness.

However, Ben explains that they “haven’t included outliers in the exercise because there exists a variety of different ways to draw box plots.”

I am not directing this to Ben particularly, but isn’t the real point of teaching box plots so that students can *read* them? And aren’t they going to have to read all the varieties in use? Drawing box plots by hand is not a CC standard, and drawing them using the Khan Academy widget is not something they will ever see again. But someone came up with a way to simulate drawing box plots with no outliers, so now it’s part of the program and teachers are going to have to teach students how to follow the steps to draw a box plot.

This is at the expense of spending time on what the consensus of math teachers on this page seems to think is important: interpreting and comparing box plots of real-world data. Presumably, from real sources with slightly different formatting standards. Because the standard doesn’t mention outliers specifically or comparing two box plots, does that mean that Khan Academy or anybody else aiming for CCSS alignment is going to skip those things? Isn’t this the kind of shallow, broad “coverage” approach with lots of rules that we are supposedly leaving behind? Why are the mechanics of sorting numbers and calculating medians being taught, drilled and tested under the name of box plots instead of the real thing? Since box plots are the only part of the 6th grade standards that prompt a discussion of what an outlier is and a precise definition, it’s not something to gloss over lightly.

Again, I don’t mean to pick on Ben and Khan too specifically–I worked with a 7th grade textbook that took the same approach, and I know they’re working from what the current texts do. I think the box plot page is going to be the page I check out first from now on when I’m trying to get a feel for a new math text!

The inequality problems still seem to be the same question asked a bunch of times. Maybe involve a number line in some of the questions?

The expression problems are ok. Especially like: __ ( __ x + 7q) is equivalent to 10x + 14q.

The function questions need a lot of editing. What is the end behavior for i) energy stored in a spring as a function of displacement ii) water left in a draining bucket as a function of time iii) force needed to keep an object on a carousel as a function of rotation speed (correct answer — as carousel rotates faster & faster there is no LIMIT to the amount of force needed). Yikes!

The “fix” to the box plot problem makes the problem even more procedural. Students now have to deal with a technicality for defining quartiles that really is not very important.

I only know what I read on their blog, etc, but my sense is that their strategy is just to seed their page with enough content to get lots of users, and then open content creation up to the masses so ANYONE can create materials (for example, Christopher could make them himself). Then the system will use data mining to figure out which ones are most effective. You can see KA’s lead dev describe this a little bit in this talk, starting at the 40:50 mark:

http://player

So maybe KA is actually ASSUMING that they don’t have the ability to create the best learning experiences, and therefore that they should focus now on building the infrastructure to allow everyone else to make content and upload it so it can be tested.

http://player

I say this as a teacher who stopped using KA last year because it was too hard for students to earn proficiencies. But I think KA is working on lots of these problems, and those infrastructure problems are frankly bigger than the issues with decimals or other content areas, because without better infrastructure, you still wouldn’t be able to do much even with better content.