# How to do a two-sided hypothesis test for two sample means

## Description

If we have two samples, $x_1,\ldots,x_n$ and $x’_1,\ldots,x’_m$, and we compute the mean of each one, we might want to ask whether the two means seem approximately equal. Or more precisely, is their difference statistically significant at a given level?

Related tasks:

- How to compute a confidence interval for a population mean
- How to do a two-sided hypothesis test for a sample mean
- How to do a one-way analysis of variance (ANOVA)
- How to do a one-sided hypothesis test for two sample means
- How to do a hypothesis test for a mean difference (matched pairs)
- How to do a hypothesis test for a population proportion

## Solution, in Julia

If we call the mean of the first sample $\bar x_1$ and the mean of the second sample $\bar x_2$, then this is a two-sided test with the null hypothesis $H_0:\bar x_1=\bar x_2$. We choose a value $0\leq\alpha\leq1$ as the probability of a Type I error (false positive, finding we should reject $H_0$ when it’s actually true).

1
2
3
4
5
6
7
8
9
10
11

# Replace these first three lines with the values from your situation.
alpha = 0.10
sample1 = [ 6, 9, 7, 10, 10, 9 ]
sample2 = [ 12, 14, 10, 17, 9 ]
# Run a one-sample t-test and print out alpha, the p value,
# and whether the comparison says to reject the null hypothesis.
using HypothesisTests
p_value = pvalue( UnequalVarianceTTest( sample1, sample2 ) )
reject_H0 = p_value < alpha
alpha, p_value, reject_H0

1

(0.1, 0.050972837418476996, true)

In this case, the $p$-value was less than $\alpha$, so the sample gives us enough evidence to reject the null hypothesis at the $\alpha=0.10$ level. The data suggest that $\bar x_1\neq\bar x_2$.

When you are using the most common value for $\alpha$, which is $0.05$ for the $95\%$ confidence interval, you can simply print out the test itself and get a detailed printout with all the information you need, thus saving a few lines of code. Note that this gives a different answer below than the one above, because above we chose to use $\alpha=0.10$, but the default below is $\alpha=0.05$.

1

UnequalVarianceTTest( sample1, sample2 )

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

Two sample t-test (unequal variance)
------------------------------------
Population details:
parameter of interest: Mean difference
value under h_0: 0
point estimate: -3.9
95% confidence interval: (-7.823, 0.02309)
Test summary:
outcome with 95% confidence: fail to reject h_0
two-sided p-value: 0.0510
Details:
number of observations: [6,5]
t-statistic: -2.4616581720814326
degrees of freedom: 5.720083530052662
empirical standard error: 1.584297951775486

Here we did not assume that the two samples had equal variance.
If in your case they do, you can use `EqualVarianceTTest()`

instead of
`UnequalVarianceTTest()`

.

Content last modified on 24 July 2023.

See a problem? Tell us or edit the source.

## Using SciPy, in Python

If we call the mean of the first sample $\bar x_1$ and the mean of the second sample $\bar x_2$, then this is a two-sided test with the null hypothesis $H_0:\bar x_1=\bar x_2$. We choose a value $0\leq\alpha\leq1$ as the probability of a Type I error (false positive, finding we should reject $H_0$ when it’s actually true). Let’s use $\alpha=0.10$ as an example.

1
2
3
4
5
6
7
8
9
10

from scipy import stats
# Replace these first three lines with the values from your situation.
alpha = 0.10
sample1 = [ 6, 9, 7, 10, 10, 9 ]
sample2 = [ 12, 14, 10, 17, 9 ]
# Run a one-sample t-test and print out alpha, the p value,
# and whether the comparison says to reject the null hypothesis.
stats.ttest_ind( sample1, sample2, equal_var=False )

1

Ttest_indResult(statistic=-2.4616581720814326, pvalue=0.05097283741847698)

The output says that the $p$-value is about $0.05097$, which is less than $\alpha=0.10$. In this case, the samples give us enough evidence to reject the null hypothesis at the $\alpha=0.10$ level. That is, the data suggest that $\bar x_1\neq\bar x_2$.

The `equal_var`

parameter tells SciPy *not* to assume that the two samples
have equal variances. If in your case they do, you can omit that parameter,
and it will revert to its default value of `True`

.

Content last modified on 24 July 2023.

See a problem? Tell us or edit the source.

## Solution, in R

If we call the mean of the first sample $\bar x_1$ and the mean of the second sample $\bar x_2$, then this is a two-sided test with the null hypothesis $H_0:\bar x_1=\bar x_2$. We choose a value $0\leq\alpha\leq1$ as the probability of a Type I error (false positive, finding we should reject $H_0$ when it’s actually true).

1
2
3
4
5
6
7
8

# Replace these first three lines with the values from your situation.
alpha <- 0.10
sample1 <- c( 6, 9, 7, 10, 10, 9 )
sample2 <- c( 12, 14, 10, 17, 9 )
# Run a one-sample t-test and print out alpha, the p value,
# and whether the comparison says to reject the null hypothesis.
t.test( sample1, sample2, conf.level=1-alpha )

1
2
3
4
5
6
7
8
9
10

Welch Two Sample t-test
data: sample1 and sample2
t = -2.4617, df = 5.7201, p-value = 0.05097
alternative hypothesis: true difference in means is not equal to 0
90 percent confidence interval:
-7.0057683 -0.7942317
sample estimates:
mean of x mean of y
8.5 12.4

Although we can deduce the answer to our question from the above output, by comparing the $p$ value with $\alpha$ manually, we can also ask R to do it.

1
2
3

# Is there enough evidence to reject the null hypothesis?
result <- t.test( sample1, sample2, conf.level=1-alpha )
result$p.value < alpha

1

[1] TRUE

In this case, the samples give us enough evidence to reject the null hypothesis at the $\alpha=0.10$ level. The data suggest that $\bar x_1\neq\bar x_2$.

Here we did not assume that the two samples had equal variance.
If in your case they do, you can pass the parameter `var.equal=TRUE`

to `t.test`

.

Content last modified on 24 July 2023.

See a problem? Tell us or edit the source.

## Topics that include this task

## Opportunities

This website does not yet contain a solution for this task in any of the following software packages.

- Excel

If you can contribute a solution using any of these pieces of software, see our Contributing page for how to help extend this website.