How to do a goodness of fit test for a multinomial experiment (in Python, using SciPy)

See all solutions.

Task

If we have historical values for multiple population proportions, plus more recent samples from those same populations, we may want to compare to see if the proportions appear to have changed. This is called a goodness of fit test for a multinomial experiment. How can we execute it?

Solution

Let’s say we have a dataset with the previous population proportions for four categories. (This is contrived data, but the code below can be used on your actual data.)

We have also taken a more recent sample and found the number of observations from it that belong to each category. We want to determine if the proportions coming from the recent sample are equal to the previous proportions.

SciPy expects that we will have two lists, one with the expected number of observations in each group (from the previous, or hypothesized proportions) and the other with the actual number of observations in each group (from the more recent sample). SciPy also expects that the total number of observations in each list is the same. We’ll create two lists below with the fake data from above, but you can replace them with your real data

1
2
3
Copy
# Replace your data in the next two lines
old_observations = [43, 62, 52, 16]
new_observations = [56, 80, 12, 25]

We set the null hypothesis to be that the proportions of each category from the recent sample are equal to the previous proportions

We choose a value $0\le \alpha \le 1$ as our Type 1 error rate. We’ll let $\alpha$ be 0.05 here.

1
2
3
Copy
# Run the Chi-square test, giving the test statistic and p-value
from scipy import stats
stats.chisquare(f_obs=new_observations, f_exp=old_observations)

1
Power_divergenceResult(statistic=44.98776977898321, pvalue=9.30824439694332e-10)

Our $p$-value is less than $\alpha$, so we have sufficient evidence to reject the null hypothesis. It does appear that the proportion of at least one of the four categories is significantly different now from what it was previously.

Content last modified on 24 July 2023.

See a problem? Tell us or edit the source.

Contributed by Elizabeth Czarniak (CZARNIA_ELIZ@bentley.edu)