Interpreting Hypothesis Tests

Interpreting the results of hypothesis tests is one of the more challenging aspects of this method for statistical inference. Let’s understand the process and address some common misconceptions.

Two possible outcomes

Given a prespecified significance level $\alpha$, there are two possible outcomes of a hypothesis test:

• If the $p$-value is less than $\alpha$, then we reject the null hypothesis $H_0$ in favor of $H_A$.

• If the $p$-value is greater than or equal to $\alpha$, we fail to reject the null hypothesis $H_0$.

Unfortunately, the latter result is often misinterpreted as accepting the null hypothesis $H_0$. While at first glance, it might seem that the statements “failing to reject $H_0$” and “accepting $H_0$” are equivalent, there actually is a subtle difference. Saying that we accept the null hypothesis $H_0$ is equivalent to stating that we think the null hypothesis $H_0$ is true. However, saying that we fail to reject the null hypothesis $H_0$ is saying something else: While $H_0$ ...