When creating a factorial design, it is important that it has adequate power to detect significant main effects and interaction effects of interest. How can we calculate the power of a two-sample $t$ test that we aim to perform in such a situation?
statsmodels, we use the
solve_power function in the
TTestIndPower class. That function embodies a relationship among five variables; you provide any four of them and it will compute the fifth to be consistent with the first four, regarding the two-sample $t$-test you plan to perform. Let’s get started by importing the package and create a
1 2 from statsmodels.stats.power import TTestIndPower analysis = TTestIndPower()
For this example, let’s say that:
- You plan to create a balanced $4\times2$ factorial experiment with 32 subjects.
- You expect the effect size for the main effect of factor A to be medium (0.25 according to Cohen’s 1988 text).
- You want to know the expected power for the test of a main effect of factor A.
- Your significance level is $\alpha=0.05$.
We proceed as follows.
1 2 3 4 5 6 7 obs = 32 # number of subjects (or observations) effect = 0.25 # effect size alpha = 0.05 # significance level ratio = 1 # ratio of the number of observations in one sample to the other # We leave power unspecified, so that solve_power will compute it for us: analysis.solve_power( effect_size=effect, power=None, nobs1=obs, ratio=ratio, alpha=alpha )
The power is 0.1663, which means that the probability of rejecting the null hypothesis when in fact it is false OR the probability of avoiding a Type II error is 0.1663.
Content last modified on 24 July 2023.
Contributed by Krtin Juneja (KJUNEJA@falcon.bentley.edu)