How to do a goodness of fit test for a multinomial experiment (in R)

See all solutions.

Task

If we have historical values for multiple population proportions, plus more recent samples from those same populations, we may want to compare to see if the proportions appear to have changed. This is called a goodness of fit test for a multinomial experiment. How can we execute it?

Solution

Let’s say we have a dataset with the previous population proportions for four categories. (This is contrived data, but the code below can be used on your actual data.)

We have also taken a more recent sample and found the number of observations from it that belong to each category. We want to determine if the proportions coming from the recent sample are equal to the previous proportions.

R expects that we will have two vectors, one with the expected number of observations in each group (from the previous, or hypothesized proportions) and the other with the actual number of observations in each group (from the more recent sample). R also expects that the total number of observations in each vector is the same. We’ll create two vectors below with the fake data from above, but you can replace them with your real data

1
2
3
4
5
6
# our fake data:
old.observations <- c(43, 62, 52, 16)
new.observations <- c(56, 80, 12, 25)
# now organized into a data frame:
categories <- c("A", "B", "C", "D")
data <- data.frame(categories, old.observations, new.observations)

We set the null hypothesis to be that the proportions of each category from the recent sample are equal to the previous proportions.

We choose a value $0 \le \alpha \le 1$ as our Type 1 error rate. We’ll let $\alpha$ be 0.05 here.

1
2
# Run the Chi-Square test, giving the test statistic and p-value
chisq.test(data$new.observations, p=data$old.observations, rescale.p=TRUE)

1
2
3
4
	Chi-squared test for given probabilities

data:  data$new.observations
X-squared = 44.988, df = 3, p-value = 9.308e-10

Our $p$-value is less than $\alpha$, so we have sufficient evidence to reject the null hypothesis. It does appear that the proportion of at least one of the four categories is significantly different now from what it was previously.

If instead you provided the population proportions as the old observations, that is, a vector of values that sum to 1, you can omit the rescale.p argument.

Content last modified on 24 July 2023.

See a problem? Tell us or edit the source.

Contributed by Elizabeth Czarniak (CZARNIA_ELIZ@bentley.edu)