# How to do a hypothesis test for a mean difference (matched pairs)

## Description

Say we have two sets of data that are not independent of each other and come from a matched-pairs experiment, $(x_1,x’_1),(x_2,x’_2),\ldots,(x_n,x’_n)$. We want to perform inference on the mean of the differences between these two samples, that is, the mean of $x_1-x’_1,x_2-x’_2,\ldots,x_n-x’_n$, called $\mu_D$. We want to determine if it is significantly different from, greater than, or less than zero (or any other hypothesized value). We can do so with a two-tailed, right-tailed, or left-tailed hypothesis test for matched pairs.

## Using SciPy, in Python

View this solution alone.

We choose a value, $0 \le \alpha \le 1$, as the Type I Error rate, and in this case we will set it to be 0.05.

We’re going to use fake fata here, but you can replace our fake data with your real data below. Because the data are matched pairs, the samples must be the same size.

1
2
3
# Replace the following example data with your real data
sample1 = [15, 10,  7, 22, 17, 14]
sample2 = [ 9,  1, 11, 13,  3,  6]


### Two-tailed test

In a two-sided hypothesis test, the null hypothesis states that the mean difference is equal to 0 (or some other hypothesized value), $H_0: \mu_D = 0$.

1
2
from scipy import stats
stats.ttest_rel(sample1, sample2, alternative = "two-sided")

1
TtestResult(statistic=2.8577380332470415, pvalue=0.03550038112896236, df=5)


Our $p$-value, 0.0355, is smaller than $\alpha$, so we have sufficient evidence to reject the null hypothesis and conclude that the mean difference between the two samples is significantly different from zero.

Note that the function above specifically tests whether the mean of $x_i-x’_i$ is zero. If we want instead to test whether it is some other value $d\neq0$, then that’s equivalent to testing whether the mean of $(x_i-d)-x’_i$ is zero. We could do so with the code below, which uses an example value of $d$. The null hypothesis is now $H_0: \mu_D=d$.

1
2
d = 6  # as an example
stats.ttest_rel([ x - d for x in sample1 ], sample2, alternative = "two-sided")

1
TtestResult(statistic=0.4082482904638631, pvalue=0.6999865427788738, df=5)


The above $p$-value is greater than $\alpha=0.05$, so we could not conclude that the mean difference is significantly different from our chosen $d=6$.

### Right-tailed test

If instead we want to test whether the mean difference is less than or equal to zero, $H_0: \mu_D\le0$, we can use a right-tailed test, as follows.

1
stats.ttest_rel(sample1, sample2, alternative = "greater")

1
TtestResult(statistic=2.8577380332470415, pvalue=0.01775019056448118, df=5)


Our $p$-value, 0.01775, is smaller than $\alpha$, so we have sufficient evidence to reject the null hypothesis and conclude that the mean difference between the two samples is significantly greater than zero.

A similar change could be made to the code above to test $H_0:\mu_D\le d$, as in the example code further above that uses $d=6$.

### Left-tailed test

If instead we want to test whether the mean difference is greater than or equal to zero, $H_0: \mu_D\ge 0$, we can use a right-tailed test, as follows.

1
stats.ttest_rel(sample1, sample2, alternative = "less")

1
TtestResult(statistic=2.8577380332470415, pvalue=0.9822498094355188, df=5)


Our $p$-value, 0.98225, is larger than $\alpha$, so we do not have sufficient evidence to reject the null hypothesis; we must continue to assume that the mean difference between the two samples is greater than or equal to zero.

A similar change could be made to the code above to test $H_0:\mu_D\ge d$, as in the example code further above that uses $d=6$.

See a problem? Tell us or edit the source.

## Solution, in R

View this solution alone.

We choose a value, $0 \le \alpha \le 1$, as the Type I Error rate, and in this case we will set it to be 0.05.

We’re going to use fake fata here, but you can replace our fake data with your real data below. Because the data are matched pairs, the samples must be the same size.

1
2
3
# Replace the following example data with your real data
sample.1 <- c(15, 10,  7, 22, 17, 14)
sample.2 <- c( 9,  1, 11, 13,  3,  6)


### Two-tailed test

In a two-sided hypothesis test, the null hypothesis states that the mean difference is equal to 0 (or some other hypothesized value), $H_0: \mu_D = 0$.

1
2
3
alpha = 0.05
t.test(sample.1, sample.2, alternative = "two.sided",
mu = 0, paired = TRUE, conf.level = 1-alpha)

1
2
3
4
5
6
7
8
9
10
Paired t-test

data:  sample.1 and sample.2
t = 2.8577, df = 5, p-value = 0.0355
alternative hypothesis: true mean difference is not equal to 0
95 percent confidence interval:
0.7033862 13.2966138
sample estimates:
mean difference
7


Our $p$-value, 0.0355, appears in the third line of the output. It is smaller than $\alpha$, so we have sufficient evidence to reject the null hypothesis and conclude that the mean difference between the two samples is significantly different from zero.

If we want instead to test whether it is some other value $d\neq0$, then just use that value as the mu parameter to the t.test function instead of zero.

### Right-tailed test

If instead we want to test whether the mean difference is less than or equal to zero, $H_0: \mu_D\le0$, we can use a right-tailed test, as follows.

1
2
t.test(sample.1, sample.2, alternative = "greater",
mu = 0, paired = TRUE, conf.level = 1-alpha)

1
2
3
4
5
6
7
8
9
10
Paired t-test

data:  sample.1 and sample.2
t = 2.8577, df = 5, p-value = 0.01775
alternative hypothesis: true mean difference is greater than 0
95 percent confidence interval:
2.06416     Inf
sample estimates:
mean difference
7


Our $p$-value, 0.01775, is smaller than $\alpha$, so we have sufficient evidence to reject the null hypothesis and conclude that the mean difference between the two samples is significantly greater than zero.

Again, you can use another value $d\neq0$ in place of mu = 0 in the code.

### Left-tailed test

If instead we want to test whether the mean difference is greater than or equal to zero, $H_0: \mu_D\ge0$, we can use a right-tailed test, as follows.

1
2
t.test(sample.1, sample.2, alternative = "less",
mu = 0, paired = TRUE, conf.level = 1-alpha)

1
2
3
4
5
6
7
8
9
10
Paired t-test

data:  sample.1 and sample.2
t = 2.8577, df = 5, p-value = 0.9822
alternative hypothesis: true mean difference is less than 0
95 percent confidence interval:
-Inf 11.93584
sample estimates:
mean difference
7


Our $p$-value, 0.9822, is larger than $\alpha$, so we do not have sufficient evidence to reject the null hypothesis; we must continue to assume that the mean difference between the two samples is greater than or equal to zero.

Again, you can use another value $d\neq0$ in place of mu = 0 in the code.