# How to do a two-sided hypothesis test for two sample means

## Description

If we have two samples, $x_1,\ldots,x_n$ and $x’_1,\ldots,x’_m$, and we compute the mean of each one, we might want to ask whether the two means seem approximately equal. Or more precisely, is their difference statistically significant at a given level?

## Solution, in Julia

View this solution alone.

If we call the mean of the first sample $\bar x_1$ and the mean of the second sample $\bar x_2$, then this is a two-sided test with the null hypothesis $H_0:\bar x_1=\bar x_2$. We choose a value $0\leq\alpha\leq1$ as the probability of a Type I error (false positive, finding we should reject $H_0$ when it’s actually true).

 1 2 3 4 5 6 7 8 9 10 11 # Replace these first three lines with the values from your situation. alpha = 0.10 sample1 = [ 6, 9, 7, 10, 10, 9 ] sample2 = [ 12, 14, 10, 17, 9 ] # Run a one-sample t-test and print out alpha, the p value, # and whether the comparison says to reject the null hypothesis. using HypothesisTests p_value = pvalue( UnequalVarianceTTest( sample1, sample2 ) ) reject_H0 = p_value < alpha alpha, p_value, reject_H0
 1 (0.1, 0.050972837418476996, true)

In this case, the $p$-value was less than $\alpha$, so the sample gives us enough evidence to reject the null hypothesis at the $\alpha=0.10$ level. The data suggest that $\bar x_1\neq\bar x_2$.

When you are using the most common value for $\alpha$, which is $0.05$ for the $95\%$ confidence interval, you can simply print out the test itself and get a detailed printout with all the information you need, thus saving a few lines of code. Note that this gives a different answer below than the one above, because above we chose to use $\alpha=0.10$, but the default below is $\alpha=0.05$.

 1 UnequalVarianceTTest( sample1, sample2 )
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Two sample t-test (unequal variance) ------------------------------------ Population details: parameter of interest: Mean difference value under h_0: 0 point estimate: -3.9 95% confidence interval: (-7.823, 0.02309) Test summary: outcome with 95% confidence: fail to reject h_0 two-sided p-value: 0.0510 Details: number of observations: [6,5] t-statistic: -2.4616581720814326 degrees of freedom: 5.720083530052662 empirical standard error: 1.584297951775486

Here we did not assume that the two samples had equal variance. If in your case they do, you can use EqualVarianceTTest() instead of UnequalVarianceTTest().

See a problem? Tell us or edit the source.

## Using SciPy, in Python

View this solution alone.

If we call the mean of the first sample $\bar x_1$ and the mean of the second sample $\bar x_2$, then this is a two-sided test with the null hypothesis $H_0:\bar x_1=\bar x_2$. We choose a value $0\leq\alpha\leq1$ as the probability of a Type I error (false positive, finding we should reject $H_0$ when it’s actually true). Let’s use $\alpha=0.10$ as an example.

 1 2 3 4 5 6 7 8 9 10 from scipy import stats # Replace these first three lines with the values from your situation. alpha = 0.10 sample1 = [ 6, 9, 7, 10, 10, 9 ] sample2 = [ 12, 14, 10, 17, 9 ] # Run a one-sample t-test and print out alpha, the p value, # and whether the comparison says to reject the null hypothesis. stats.ttest_ind( sample1, sample2, equal_var=False )
 1 Ttest_indResult(statistic=-2.4616581720814326, pvalue=0.05097283741847698)

The output says that the $p$-value is about $0.05097$, which is less than $\alpha=0.10$. In this case, the samples give us enough evidence to reject the null hypothesis at the $\alpha=0.10$ level. That is, the data suggest that $\bar x_1\neq\bar x_2$.

The equal_var parameter tells SciPy not to assume that the two samples have equal variances. If in your case they do, you can omit that parameter, and it will revert to its default value of True.

See a problem? Tell us or edit the source.

## Solution, in R

View this solution alone.

If we call the mean of the first sample $\bar x_1$ and the mean of the second sample $\bar x_2$, then this is a two-sided test with the null hypothesis $H_0:\bar x_1=\bar x_2$. We choose a value $0\leq\alpha\leq1$ as the probability of a Type I error (false positive, finding we should reject $H_0$ when it’s actually true).

 1 2 3 4 5 6 7 8 # Replace these first three lines with the values from your situation. alpha <- 0.10 sample1 <- c( 6, 9, 7, 10, 10, 9 ) sample2 <- c( 12, 14, 10, 17, 9 ) # Run a one-sample t-test and print out alpha, the p value, # and whether the comparison says to reject the null hypothesis. t.test( sample1, sample2, conf.level=1-alpha )
 1 2 3 4 5 6 7 8 9 10 Welch Two Sample t-test data: sample1 and sample2 t = -2.4617, df = 5.7201, p-value = 0.05097 alternative hypothesis: true difference in means is not equal to 0 90 percent confidence interval: -7.0057683 -0.7942317 sample estimates: mean of x mean of y 8.5 12.4

Although we can deduce the answer to our question from the above output, by comparing the $p$ value with $\alpha$ manually, we can also ask R to do it.

 1 2 3 # Is there enough evidence to reject the null hypothesis? result <- t.test( sample1, sample2, conf.level=1-alpha ) result$p.value < alpha  1 [1] TRUE In this case, the samples give us enough evidence to reject the null hypothesis at the$\alpha=0.10$level. The data suggest that$\bar x_1\neq\bar x_2\$.

Here we did not assume that the two samples had equal variance. If in your case they do, you can pass the parameter var.equal=TRUE to t.test.