Jul 032013

This is a two part lesson, an experiment and simulation that together meets the CCSSM S.IC.B.4-6 and explains AP Stats concepts as well:


This exercise is one we did at the SilverState AP Institute with Josh Tabor, but the nice this is that if you do the first part it fits in Alg 2 nicely, but you can extend it to AP Stats as well.

It is LONG, with lots of pictures to explain how to use the applet so I will put the post after a break:

So, let’s do an experiment. The one we did was with soda (with and without caffeine) and heart rate, but it can be anything. How about walking in place for 1 minute and effect on heart rate? Or memory games, or any number of creative experiments. (for those of you like me who lack creativity, thankfully there are sites like these: http://brainu.org/classroom-experiments or you could use the “Meaningful Words from the NCTM “Reasoning and Sensemaking” book or this Clothespin Lab -pdf)


Okay, so we have an experiment picked out. I am going to use walking in place for 1 minute and the effect on heart rate in my example, but you should substitute your own better example.

1. Random assignment of treatment and control.

Very important to randomly assign who is going to be in the treatment or the control. After all, if we don’t do that, we don’t have a good experiment, and for a lesser reason, it is there in S.IC.B.5. I will use a well shuffled deck of cards and the Red cards will be treatment and the Black cards control. You could use a die and even / odd, or a coin flip and head / tail.

2. Establish the “before” and take the pulse. An easy way is to have the learners find their pulse, and then the teacher uses a clock to measure 30 seconds. Have the learners take their measurement and times by 2. Learners then write down this number.

3. Next, do the experiment. The control group stands but does not walk, while the treatment group walks in place for 1 minute or 30 seconds, you decide.

4. Finally, take have everyone take their pulse again using 30 seconds. [Teacher note: if you want to make sure you don’t have lots of negatives when the subtraction is done, use 28-9 seconds for the first measurement and 31 to 32 seconds for the second measurement. Yes it is cheating, but it helps them learn the stats instead of wondering why pulse rates dropped. It can happen.]

5. Learners write down this number and they subtract the two numbers.

6. Teacher asks for this number and we put it on the board as well create two dotplots, one dotplot for control, one dotplot for treatment.

7. We find the value for: Mean(treatment) – Mean(control) and write this number down.

We now have 2 choices:

A. Walking in place has an impact on heart rate.

B. Walking in place has no impact on heart rate and the increase we saw was due to random assignment.

Do we have convincing evidence for one or the other of these to statements, and how do we make sure it is CONVINCING?


The simulation is where the rubber meets the road on the CCSSM S.IC.B.4-6. Here is how we decide if the random assignment is the key. [Very Important Note: while the learners are doing this, make sure you have your data typed into excel for later use in two columns. Column 1 says standing or walking and column 2 has the difference value.]

1. Hand out enough 3×5 index cards so the learners can write down all the changes in heart rate, one per each card. If there are 30 pp in the room taking part, each learner needs 30 cards.

2. Next, shuffle the cards.

3. After shuffling the first 15 will be the results of the “control” and the next 15 will be the results of the “treatment”.

4. Calculate the value for: Mean(treatment) – Mean(control)

5. Graph it on a dotplot.

6. Do steps 2-5 again.

7. Repeat

Each cycle of 2-5 is a random simulation of possible values of the differences, if the increase in heart rate was due to random assignment alone. But we need several hundred of the simulations in order to decide if the mean difference WE got from the experiment above is CONVICING evidence.

Enter technology, specifically this site: http://lock5stat.com/statkey/ I am going to give step by step instructions with pictures on how to use this site to do a simulation for our data:

First, we will use this link found on the right side:


We are going to edit the data, and put in OUR data instead of theirs.


This box pops up when you click the “Edit Data” button, and now just copy and paste our data from excel (remember ctrl-C to copy and ctrl-V to paste). Highlight the data that is in the box, delete it, and paste our values from excel. That is all it takes!



Using the “Generate 1 Sample” button will give us exactly what the class did, one sample where we randomized the values. Notice the dot plot under Original Sample should look A LOT like the one you had on the board.


Well, we want to do more than 1 sample, so let’s do 10 more.


Still nothing that looks interesting, how about we do 1000 more?


AHA! Now we have something we can use to evaluate whether the value we got in class is unusually large or not. Where does the value we first got appear on this scale? Let’s say we got a mean increase of .83 [the numbers will be scaled to what the class data was, this picture clearly does not reflect the class data]. Is .83 CONVINCING? I would say yes. It is only around 28/1011 samples that had a value of .83, so that is highly unusual.

What about a value of .5? No, that would not be unusual.

And that is the value of a simulation to evaluate whether or not we have CONVINCING evidence. For Algebra 2, this is where we would stop. This absolutely meets and extends the CCSS S.IC.B.4-6.

For AP Stats, it is also the way a p-value is calculated!

That is one way to extend this to AP Stats! Now we have a simulation to show us what the p-value actually MEANS, not just a number on a page. I think a simulation like this would go a LONG way to helping our AP learners understand p-value and probability.

Extending this to AP Stats would mean we spend more time discussing what this distribution means, and what the simulation is doing. Depending on when we do this, a discussion of t vs. z is appropriate, but I anticipate doing this exercise much earlier in the year, way before we get into those concepts.

Another way to extend this exercise is to block it on male / female. Calculate the mean of the males and the mean of the females and then use that difference as a “handicap” for the larger value. That is the real purpose in blocking. And yes, by handicap I mean exactly what is meant in golf or bowling. Adjust the larger value down by that difference and then compare.

  2 Responses to “APStats & Alg2: Using simulation to evaluate an experiment”

  1. […] Finally, Glenn uses the power of simulation to evaluate experiments in his post, 'Using simulation to evaluate an experiment'. […]

  2. I’ve modified this to use paper basketball (blinded vs. eyes open) and I’m planning to use it as an intro activity for most of the first day of class. I typed up my notes (https://docs.google.com/a/byron.k12.mn.us/document/d/1mJravRxsjkV3FKKE0BHfc6bRTpia78jT1Ba2jYvJx1g/edit) as a rough guide of how I’m planning to lead it. Thanks for sparking the idea!

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>