How to compute the power of a test comparing two population means (in R)

See all solutions.

Task

When creating a factorial design, it is important that it has adequate power to detect significant main effects and interaction effects of interest. How can we calculate the power of a two-sample $t$ test that we aim to perform in such a situation?

Related tasks:

• How to choose the sample size in a study with two population means

Solution

We use the power.t.test function in R. It embodies a relationship among five variables; you provide any four of them and it will compute the fifth to be consistent with the first four, regarding the two-sample $t$-test you plan

For this example, let’s say that:

• You plan to create a balanced $4\times 2$ factorial experiment with 32 subjects.
• You wish to be able to detect a difference
• You want to know the expected power for the test of a main effect of factor A.
• Your significance level is $\alpha =0.05$.

We proceed as follows.

1
2
3
4
5
6
7
8
9
10
Copy
# install.packages('pwr') # if you have not already installed it
library(pwr)

obs <- 32       # number of subjects (or observations)
effect <- 0.25  # effect size 
alpha <- 0.05   # significance level
ratio <- 1      # ratio of the number of observations in one sample to the other

# We leave power unspecified, so that power.t2n.test will compute it for us:
pwr.t2n.test(n1=obs, n2=obs, d=effect, sig.level=alpha, power=NULL)

1
2
3
4
5
6
7
8
     t test power calculation 

             n1 = 32
             n2 = 32
              d = 0.25
      sig.level = 0.05
          power = 0.1662985
    alternative = two.sided

The power is 0.1663, which means that the probability of rejecting the null hypothesis when in fact it is false OR the probability of avoiding a Type II error is 0.1663.

Content last modified on 24 July 2023.

See a problem? Tell us or edit the source.

Contributed by Nathan Carter (ncarter@bentley.edu)