Precise and Shorthand Interpretation

Let’s return our attention to 95% confidence intervals. The precise and mathematically correct interpretation of a 95% confidence interval is a little long-winded:

• Precise interpretation: If we repeated our sampling procedure a large number of times, we expect about 95% of the resulting confidence intervals to capture the value of the population parameter.

This is what we observed. Our confidence interval construction procedure is 95% reliable. That is to say, we can expect our confidence intervals to include the true population parameter about 95% of the time.

A common but incorrect interpretation is that there’s a 95% probability that the confidence interval contains $p$. Each of the confidence intervals either does or doesn’t contain $p$. In other words, the probability is either a 1 or a 0.

So, if the 95% confidence level only relates to the reliability of the confidence interval construction procedure and not to a given confidence interval itself, what insight can be derived from a given confidence interval? For example, going back to the pennies example, we found that the percentile method 95% confidence interval for $\mu$ was (1991, 1999), whereas the standard error method 95% confidence interval was (1991, 1999). What can be said about these two intervals?

Loosely speaking, we can think of these intervals as our best guess of a plausible range of values for the mean year $\mu$ of all US pennies. For the rest of this course, we’ll use the following shorthand summary of the precise interpretation.

• Shorthand interpretation: We are 95% “confident” that a 95% confidence interval captures the value of the population parameter.

We use quotation marks around confident to emphasize that while 95% relates to the reliability of our confidence interval construction procedure, ultimately, a constructed confidence interval is our best guess of an interval that contains the population parameter. In other words, it’s our best net.

So returning to our pennies example and focusing on the percentile method, we are 95% “confident” that the true mean year of the pennies in circulation in 2019 is somewhere between 1991 and 1999.

Width of confidence intervals

Now that we know how to interpret confidence intervals, let’s go over some factors that determine their width.

Impact of confidence level

One factor that determines confidence interval widths is the prespecified confidence level. For example, we compared the widths of 95% and 80% confidence intervals and observed that the 95% confidence intervals were wider. The quantification of the confidence level should match what many expect of the word “confident.” In order to be more confident in our best guess of a range of values, we need to widen the range of values.

To elaborate on this, imagine we want to guess the forecasted high temperature in Seoul, South Korea, on August 15th. Given Seoul’s temperate climate with four distinct seasons, we could say somewhat confidently that the high temperature would be between 50°F–95°F (10°C–35°C). However, if we wanted a temperature range we were absolutely confident about, we would need to widen it.

We need this wider range to allow for the possibility of anomalous weather, like a cold spell or an extreme heat wave. So a range of temperatures we can be near certain about will be between 32°F–110°F (0°C–43°C). On the other hand, if we can tolerate being a little less confident, we can narrow this range to between 70°F–85°F (21°C–30°C).

Let’s revisit our sampling bowl and compare 10 * 3 = 30 confidence intervals for $p$ based on three different confidence levels that are 80%, 95%, and 99%.

Specifically, we’ll first take 30 different random samples of size $n$ = 50 balls from the bowl. Then, we’ll construct 10 percentile-based confidence intervals using each of the three different confidence levels.

Finally, we’ll compare the widths of these intervals. We visualize the resulting confidence intervals in the figure below along with a vertical line marking the true value of $p$= 0.375.