Power tutorial

1. Initially, the applet shows the hypothesised distribution of the sample mean $\overline{X}$ under the null hypothesis

$H_0: µ = µ_0$

with a significance level of $\alpha$ = 0.05, sample size of n = 3 and standard deviation $\sigma_X = 1$. The threshold for the test is also shown (observed sample means beyond this point lead to rejection of the null hypothesis).

(a) If you change the significance level $\alpha$ (all else remaining constant), how would the display change? Check your thoughts by changing $\alpha$ to 0.01 or 0.1.

(b) What if you change the sample size, n – how does the display change? Check your thoughts by varying the sample size, this time holding alpha constant.

(c) What other factor affects the position of the threshold? See if you can check your answer using the applet.

2. Reset the sample size to 3, $\alpha = 0.05$ and $\sigma = 1$. Click the ‘Show true distribution’ checkbox to display the true distribution of $\overline{X}$ with true mean µ.

(a) Which area under the true distribution corresponds to making a Type II error? Point to it on the screen.

(b) Which area under the true distribution gives the power of the test?

Click the ‘Show Type II error region’ and ‘Show power’ checkboxes to see if your thoughts were correct.

(c) If you move the true mean µ further away from/closer to $\mu_0$, how would this affect the Type I error rate, Type II error rate, and power of the test?

Check your thoughts by dragging the point labelled Alternate µ.

Note: The effect size is the standardized difference between the alternate $\mu$ and $\mu_0$. That is

In our example $\sigma_X = 1$ so the effect size reduces to the difference $(\mu – \mu_0)/1 = \mu – \mu_0$.

3. If you change the sample size (Type I error rate and difference remaining constant), how and why would this affect the Type II error rate and power of the test? Check your thoughts by using the slider to vary the sample size between 1 and 5.

4. What if the significance level $\alpha$ was changed (while µ remains constant)? Check your thoughts by setting the sample size at a fixed level and then varying $\alpha$.

5. What if $\sigma$ changes (while the sample size, $\alpha$ and µ remain fixed)? What effect does this have on the power of the test? Check using the applet.

In Summary:

Based on your exploration to date, list four measures that affect the power of a test. Describe how they affect the power.

Click here to reveal the answer

• Increasing the sample size increases the power.
• Increasing the significance level $\alpha$ increases the power.
• Increasing the standard deviation decreases the power.
• Increasing the difference in means increases the power.

Extension questions

E1. Try to answer each of the following questions without using the applet first, and then check your answer with the applet.

(a) What is the smallest possible power of the test? When does it occur?

Click here to reveal the answer
The smallest possible power is the significance level $\alpha$, which occurs when $\mu = \mu_0$.

(b) Is it possible to have a power of 0? Explain.

Click here to reveal the answer
No, because there is always a non-zero area under the tails of the distributions outside the red lines.

(c) Is there a maximum possible power of the test? When does it occur?

Click here to reveal the answer
The upper bound for the power is 1. As the alternate mean $\mu$ moves further and further from the hypothesised mean $\mu_0$ the power gets closer and closer to 1.

(d) When is the power 0.5? Try to think it through before experimenting with the applet.

In summary:

What would the graph of power vs. difference in means look like?

If a larger sample was available for the test, how would the shape of the graph change, if at all?