# How to compute the power of a test comparing two population means

## Description

When creating a factorial design, it is important that it has adequate power to detect significant main effects and interaction effects of interest. How can we calculate the power of a two-sample $t$ test that we aim to perform in such a situation?

Related tasks:

## Using statsmodels, in Python

From `statsmodels`

, we use the `solve_power`

function in the `TTestIndPower`

class. That function embodies a relationship among five variables; you provide any four of them and it will compute the fifth to be consistent with the first four, regarding the two-sample $t$-test you plan to perform. Let’s get started by importing the package and create a `TTestIndPower`

object.

1
2

from statsmodels.stats.power import TTestIndPower
analysis = TTestIndPower()

For this example, let’s say that:

- You plan to create a balanced $4\times2$ factorial experiment with 32 subjects.
- You expect the effect size for the main effect of factor A to be medium (0.25 according to Cohen’s 1988 text).
- You want to know the expected power for the test of a main effect of factor A.
- Your significance level is $\alpha=0.05$.

We proceed as follows.

1
2
3
4
5
6
7

obs = 32 # number of subjects (or observations)
effect = 0.25 # effect size
alpha = 0.05 # significance level
ratio = 1 # ratio of the number of observations in one sample to the other
# We leave power unspecified, so that solve_power will compute it for us:
analysis.solve_power( effect_size=effect, power=None, nobs1=obs, ratio=ratio, alpha=alpha )

1

0.1662985260871502

The power is 0.1663, which means that the probability of rejecting the null hypothesis when in fact it is false OR the probability of avoiding a Type II error is 0.1663.

Content last modified on 24 July 2023.

See a problem? Tell us or edit the source.

## Solution, in R

We use the `power.t.test`

function in R. It embodies a relationship among
five variables; you provide any four of them and it will compute the fifth to
be consistent with the first four, regarding the two-sample $t$-test you plan

For this example, let’s say that:

- You plan to create a balanced $4\times2$ factorial experiment with 32 subjects.
- You wish to be able to detect a difference
- You want to know the expected power for the test of a main effect of factor A.
- Your significance level is $\alpha=0.05$.

We proceed as follows.

1
2
3
4
5
6
7
8
9
10

# install.packages('pwr') # if you have not already installed it
library(pwr)
obs <- 32 # number of subjects (or observations)
effect <- 0.25 # effect size
alpha <- 0.05 # significance level
ratio <- 1 # ratio of the number of observations in one sample to the other
# We leave power unspecified, so that power.t2n.test will compute it for us:
pwr.t2n.test(n1=obs, n2=obs, d=effect, sig.level=alpha, power=NULL)

1
2
3
4
5
6
7
8

t test power calculation
n1 = 32
n2 = 32
d = 0.25
sig.level = 0.05
power = 0.1662985
alternative = two.sided

The power is 0.1663, which means that the probability of rejecting the null hypothesis when in fact it is false OR the probability of avoiding a Type II error is 0.1663.

Content last modified on 24 July 2023.

See a problem? Tell us or edit the source.

## Topics that include this task

## Opportunities

This website does not yet contain a solution for this task in any of the following software packages.

- Excel
- Julia

If you can contribute a solution using any of these pieces of software, see our Contributing page for how to help extend this website.