Common Mistakes When Interpreting P-values in Hypothesis Tests

📚 Quick Study Guide

• 🔢 The p-value is the probability of obtaining results as extreme as, or more extreme than, the observed results, assuming the null hypothesis is true.
• 📉 A small p-value (typically $\leq 0.05$) suggests strong evidence against the null hypothesis, so you reject it.
• 📈 A large p-value ($> 0.05$) suggests weak evidence against the null hypothesis, so you fail to reject it.
• 🚫 The p-value is NOT the probability that the null hypothesis is true.
• ⚖️ The p-value is NOT the probability that your results are due to chance alone.
• 🧪 Statistical significance does not necessarily imply practical significance.
• 📊 Always consider the context of your study and the potential for confounding variables.

Practice Quiz

1. Question 1: What does a p-value of 0.03 indicate?
   1. It indicates a 3% chance that the null hypothesis is true.
   2. It indicates a 3% chance of making a Type I error if the null hypothesis is rejected.
   3. It indicates a 3% chance that the alternative hypothesis is false.
   4. It indicates that the null hypothesis is true 97% of the time.
2. Question 2: Which of the following is a common misinterpretation of p-values?
   1. The p-value represents the probability of observing the data given the null hypothesis is true.
   2. The p-value quantifies the strength of evidence against the null hypothesis.
   3. The p-value measures the practical significance of the results.
   4. The p-value is used to determine the statistical significance of the results.
3. Question 3: If a study yields a p-value of 0.15, what is the correct interpretation?
   1. The null hypothesis is true.
   2. There is strong evidence against the null hypothesis.
   3. There is weak evidence against the null hypothesis.
   4. The alternative hypothesis is true.
4. Question 4: What is the significance level ($\alpha$) typically used in hypothesis testing?
   1. 0.01
   2. 0.05
   3. 0.10
   4. 0.50
5. Question 5: A researcher obtains a p-value of 0.001. What does this suggest?
   1. The null hypothesis is likely true.
   2. The alternative hypothesis is likely false.
   3. There is strong evidence against the null hypothesis.
   4. The sample size is too small.
6. Question 6: What does it mean to 'fail to reject the null hypothesis'?
   1. The null hypothesis is proven true.
   2. There is not enough evidence to reject the null hypothesis.
   3. The alternative hypothesis is true.
   4. A Type I error has occurred.
7. Question 7: A study finds a statistically significant result (p < 0.05), but the effect size is very small. What should the researcher conclude?
   1. The results are practically significant and important.
   2. The results are statistically significant but may not be practically significant.
   3. The null hypothesis should be accepted.
   4. A Type II error has occurred.