Confidence intervals, hypothesis testing and p-values tutorial – an amiable correspondence
Purpose of the applet
To make explicit the correspondence between inference based on confidence intervals (CIs) and formal hypothesis testing using $p$-values.
About the applet
The applet initially shows a 95% confidence interval around a sample mean $\bar{x}$.
The null distribution and a graph of $p$ vs. $\mu_0$ can be displayed by enabling the checkboxes at top-right.
The hypothesised mean $\mu_0$ can be varied by dragging the point on the lower axis.
It can also be set by entering a desired effect $(\mu_0 – \bar{x})$ or $p$-value in the text boxes at the bottom.
The level of confidence is fixed at 95%; the standard error $\left( \frac{\sigma}{\sqrt{n}}\right)$ and sample mean $(\bar{x})$ are also fixed.
Tutorial
Step 1:
Begin with the default setting $(\mu_0 = \bar{x})$.
A 95% CI is displayed on the screen. Based on this CI, what values of $\mu_0$ would be considered as plausible?
Drag $\mu_0$ to check your thoughts.
Step 2:
Check the box ‘Show null distribution‘ and use the slider to investigate the following:
What can you say about the value of $p$ when the hypothesized mean ($\mu_0$) is:
- inside the CI? The $p$-value is between Click here to check your answer
- outside the CI? The $p$-value is between Click here to check your answer
- at the edge of the CI? The $p$-value is and $|\mu_0 – \bar{x}|$ is Click here to check your answer
What is the maximum possible value of p? When does it occur? Click here to check your answer
How quickly does $p$ change as $(\mu_0 – \bar{x})$ changes, if $\mu_0$ is
- close to $\bar{x}$? Click here to check your answer
- close the edge of the CI? Click here to check your answer
- well outside the CI? Click here to check your answer
If you were to plot $p$ against $(\mu_0 – \bar{x})$ what do you think the graph would look like?
Check the box ‘Show p curve‘ and see if your thoughts were correct.
There is a dotted line on the $p$ curve … what does it represent? Click here to check your answer
Extensions:
what if the level of confidence changes?
Scenario 1: Suppose we had constructed a 99% CI rather than a 95% CI.
(a) If hypothesized mean is at the extreme of the CI then:
- the $p$-value would be Click here to check your answer
- $|\mu_0 – \bar{x}|$ would be Click here to check your answer
(b) If $|\mu_0 – \bar{x}| = 1.96 \frac{\sigma}{\sqrt{n}}$ then $p$-value is closest to (choose one): 0.1, 0.05, 0.005, 0.003, 0.001
(Estimate first and then use the applet to check your answer.)
Click here to check your answer
(c) If $|\mu_0 – \bar{x}| = 2.8 \frac{\sigma}{\sqrt{n}}$ then $p$-value is closest to (choose one): 0.1, 0.05, 0.005, 0.003, 0.001
(Estimate first and then use the applet to check your answer.)
Click here to check your answer
(d) If the hypothesised mean $\mu_0$ equals the sample mean $\bar{x}$ then $p = $
Click here to check your answer
(e) Would the plot of $p$ vs. $(\mu_0 – \bar{x})$ be different in this scenario?
Click here to check your answer
Scenario 2: Suppose we had constructed a 90% CI rather than a 95% CI.
(a) If hypothesized mean is at the extreme of the CI then:
- the $p$-value would be Click here to check your answer
- $|\mu_0 – \bar{x}|$ would be Click here to check your answer
(b) If $|\mu_0 – \bar{x}| = 1.96 \frac{\sigma}{\sqrt{n}}$ then $p$-value is closest to (choose one): 0.1, 0.05, 0.005, 0.003, 0.001
(Estimate first and then use the applet to check your answer.)
Click here to check your answer
(c) If $|\mu_0 – \bar{x}| = 2.8 \frac{\sigma}{\sqrt{n}}$ then $p$-value is closest to (choose one): 0.1, 0.05, 0.005, 0.003, 0.001
(Estimate first and then use the applet to check your answer.)
Click here to check your answer
(d) If the hypothesised mean $\mu_0$ equals the sample mean $\bar{x}$ then $p = $
Click here to check your answer
(e) Would the plot of $p$ vs. $(\mu_0 – \bar{x})$ be different in this scenario?
Click here to check your answer