Type I and Type II error are concepts my students have struggled with year after year. This school year, I decided to do something different in AP Stats.
KNEB news story: "Rabies Outbreak Soars in Goshen County"
(this particular county is very close to where we live in western Nebraska)
The news story above was posted the night before I was slated to introduce Type I and Type II error for the first time. The text of the news story caught my eye because something, literally, didn't add up.
...since February 7th of this year, out of 19 dead skunks collected, 16 tested positive for rabies. They also had 1 red fox test positive for the disease... Mills will be retiring next month and has been at the lab since 1984 and has never seen, even in an outbreak situation, this high of a percentage rate of positive samples at 89.5%...
It is interesting in the news story that the reported percentage, 89.5%, does not match the fraction 16/19 (which is approximately 84.2%). The 89.5% figure is a computational error. The person computing it took included the rabid red fox, despite the fact the red fox is a different species, and also did not increase the denominator by one. The 89.5% comes from the fraction 17/19, which is obviously not valid. (We had a nice class discussion over whether this was a computational error or a purposeful error meant to sensationalize the news story.)
When I first saw this story, I decided to scrap what I had planned for the day in AP Stats and made this worksheet in the planning period I had right before class.
I spent about 45 minutes trolling the internet - in particular, the USDA and CDC websites - trying to come up with count data for how many animals are tested for rabies in Wyoming in a given year. My searches took me here and here. I found data from 2010. 32 wild animals tested for rabies in Wyoming in 2010 - 2 were cattle, 12 were bats, and 20 were skunks. I could not locate more recent data or 2010 data on the total number of animals tested, which posed a problem, because we are really interested in the total number of animals tested to inform us on what proportion of rabies cases we expect. I wanted to reverse engineer the actual percentage of rabid skunks we expect in Wyoming... how high does the percentage of positive tests have to be before we release such a warning to the public? 50%? 60%? 75%?
[side note: I learned a great deal about how rabies spreads and which animals are typically affected. See the map below.]
U.S. map showing which animal is the most frequent rabies carrier by region [source: CDC]
I asked the students to use their TI-84 calculator and, through trial and error, to determine the population proportion value p (the actual proportion of rabid skunks) that would lead to NOT rejecting the null hypothesis for each alpha level. Here's a slide with the work a student wrote on the board:
I had students use trial and error, with the TI-84 one-proportion z-test, to determine what the decision would be (whether we would reject or fail to reject the null hypothesis) for different values of the assumed population proportion p.
For example, with an alpha level of .05 and assuming the population proportion of all skunks that have rabies is 50%, we would reject the null - the population proportion being 50% and conclude in favor of the alternative, that the proportion of skunks with rabies is likely higher than 50% based on the sample, since we expect to see a sample this extreme (16 out of 19 skunks rabid) only 14 times out of every 10,000 samples due to chance alone.
Through trial and error, my students found we would not reject the null hypothesis given a sample of 16 out of 19 skunks being rabid if the actual proportion of rabid skunks was 66.37% or higher. 2/3 of all skunks being rabid is pretty scary to think about, but if we are trying to determine why the USDA released a rabies warning, this is useful information for us.
I had students formulate the idea of a Type I error (mistakenly rejecting a true null hypothesis) and a Type II error (mistakenly failing to reject a false null hypothesis) in the context of this problem through discussion.
Type I error would mean the proportion of skunks with rabies is greater than the assumed proportion when in fact the assumed proportion is true. This would mean potentially raising a false alarm, or releasing an outbreak alert when one is not needed.
Type II error would mean concluding the proportion of skunks with rabies is the assumed proportion when in fact the actual proportion of skunks with rabies is higher. This would mean not alerting the public to a potential outbreak when in fact an outbreak is actually going on.
Students collectively agreed Type II error is more serious in this setting. The students discovered through this activity the decision they should make about the alpha level - maximizing the chance of a Type I error in favor of decreasing the chance of a Type II error.
My students breezed through the exam covering Type I and Type II error for the first time in my career. 🙂