Confidence intervals, hypothesis testing and p-values - an amiable correspondence

Tutorial

Purpose of the applet

To make explicit the correspondence between inference based on confidence intervals (CIs) and formal hypothesis testing using $p$-values.

About the applet

The applet initially shows a 95% confidence interval around a sample mean $\bar{x}$.
The null distribution and a graph of $p$ vs. $\mu_0$ can be displayed by enabling the checkboxes at top-right.

The hypothesised mean $\mu_0$ can be varied by dragging the point on the lower axis. It can also be set by entering a desired effect $(\mu_0 - \bar{x})$ or $p$-value in the text boxes at the bottom.

The level of confidence is fixed at 95%; the standard error $\left( \frac{\sigma}{\sqrt{n}}\right)$ and sample mean $(\bar{x})$ are also fixed.

Tutorial

Step 1: Begin with the default setting $(\mu_0 = \bar{x})$.
A 95% CI is displayed on the screen. Based on this CI, what values of $\mu_0$ would be considered as plausible?

Drag $\mu_0$ to check your thoughts.

Step 2: Check the box 'Show null distribution' and use the slider to investigate the following:

What can you say about the value of $p$ when the hypothesized mean ($\mu_0$) is:

What is the maximum possible value of p? When does it occur? Click here to check your answer$p = 1$, which occurs when $\mu_0 = \bar{x}$

How quickly does $p$ change as $(\mu_0 - \bar{x})$ changes, if $\mu_0$ is

If you were to plot $p$ against $(\mu_0 - \bar{x})$ what do you think the graph would look like?

Check the box 'Show p curve' and see if your thoughts were correct.

There is a dotted line on the $p$ curve ... what does it represent? Click here to check your answerIt is the significance level $\alpha$, which is the $p$ threshold for the 2-sided hypothesis test.

Extensions:

what if the level of confidence changes?

Scenario 1: Suppose we had constructed a 99% CI rather than a 95% CI.

(a) If hypothesized mean is at the extreme of the CI then:

(b) If $|\mu_0 - \bar{x}| = 1.96 \frac{\sigma}{\sqrt{n}}$ then $p$-value is closest to (choose one): 0.1, 0.05, 0.005, 0.003, 0.001
(Estimate first and then use the applet to check your answer.)
Click here to check your answer0.05

(c) If $|\mu_0 - \bar{x}| = 2.8 \frac{\sigma}{\sqrt{n}}$ then $p$-value is closest to (choose one): 0.1, 0.05, 0.005, 0.003, 0.001
(Estimate first and then use the applet to check your answer.)
Click here to check your answer0.005

(d) If the hypothesised mean $\mu_0$ equals the sample mean $\bar{x}$ then $p = $ Click here to check your answer1

(e) Would the plot of $p$ vs. $(\mu_0 - \bar{x})$ be different in this scenario? Click here to check your answerThe plot would be the same, but the threshold line would move down to $p = 0.01$.


Scenario 2: Suppose we had constructed a 90% CI rather than a 95% CI.
(a) If hypothesized mean is at the extreme of the CI then:

(b) If $|\mu_0 - \bar{x}| = 1.96 \frac{\sigma}{\sqrt{n}}$ then $p$-value is closest to (choose one): 0.1, 0.05, 0.005, 0.003, 0.001
(Estimate first and then use the applet to check your answer.)
Click here to check your answer0.05

(c) If $|\mu_0 - \bar{x}| = 2.8 \frac{\sigma}{\sqrt{n}}$ then $p$-value is closest to (choose one): 0.1, 0.05, 0.005, 0.003, 0.001
(Estimate first and then use the applet to check your answer.)
Click here to check your answer0.005

(d) If the hypothesised mean $\mu_0$ equals the sample mean $\bar{x}$ then $p = $ Click here to check your answer1

(e) Would the plot of $p$ vs. $(\mu_0 - \bar{x})$ be different in this scenario? Click here to check your answerThe plot would be the same, but the threshold line would move up to $p = 0.1$.