Two-Sample Tests | StatsMasters

Comparing Two Groups

Two-sample tests help answer questions like:

Is treatment A better than treatment B?
Do men and women differ in some measurement?
Did scores improve after an intervention?

Study Design	Test
Two independent groups	Independent t-test
Same subjects, two measurements	Paired t-test
Two proportions	Two-proportion z-test

Independent Two-Sample t-Test

Compares means of two separate groups of subjects.

Independent t-Test

$t = \frac{(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)_0}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$

Usually testing H₀: μ₁ - μ₂ = 0, so:

$t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$

Degrees of Freedom

Welch’s approximation (unequal variances assumed):

$df = \frac{\left(\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}\right)^2}{\frac{(s_1^2/n_1)^2}{n_1-1} + \frac{(s_2^2/n_2)^2}{n_2-1}}$

Conservative approach: df = min(n₁ - 1, n₂ - 1)

Independent t-Test

Question: Do two teaching methods produce different test scores?

Group 1 (Method A): n₁ = 25, x̄₁ = 78, s₁ = 8 Group 2 (Method B): n₂ = 30, x̄₂ = 72, s₂ = 10 α = 0.05, two-tailed

Hypotheses:

H₀: μ₁ = μ₂ (no difference)
H₁: μ₁ ≠ μ₂ (different)

Standard error: $SE = \sqrt{\frac{8^2}{25} + \frac{10^2}{30}} = \sqrt{2.56 + 3.33} = \sqrt{5.89} = 2.43$

Test statistic: $t = \frac{78 - 72}{2.43} = \frac{6}{2.43} = 2.47$

df (conservative) = min(24, 29) = 24 Critical t = ±2.064

Decision: Since 2.47 > 2.064, reject H₀

Conclusion: Evidence suggests the methods produce different results.

Pooled vs Unpooled Variance

Method	When to Use
Pooled (equal variances)	When σ₁ = σ₂ can be assumed
Unpooled (Welch’s)	When σ₁ ≠ σ₂ or unsure

Pooled Variance

$s_p^2 = \frac{(n_1-1)s_1^2 + (n_2-1)s_2^2}{n_1 + n_2 - 2}$

$SE = s_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}$

df = n₁ + n₂ - 2

Paired t-Test

Use when the same subjects are measured twice or subjects are matched in pairs.

Paired t-Test

Calculate differences: $d_i = x_{1i} - x_{2i}$

$t = \frac{\bar{d} - 0}{s_d / \sqrt{n}}$

df = n - 1 (where n = number of pairs)

Paired t-Test

Question: Does a training program improve typing speed?

10 people measured before and after training:

Person	Before	After	Difference (d)
1	45	52	7
2	38	41	3
3	55	60	5
4	42	49	7
5	50	53	3
6	35	42	7
7	48	51	3
8	41	47	6
9	52	58	6
10	44	48	4

Statistics: n = 10, d̄ = 5.1, s_d = 1.73

Hypotheses:

H₀: μ_d = 0 (no improvement)
H₁: μ_d > 0 (improvement)

Test statistic: $t = \frac{5.1 - 0}{1.73 / \sqrt{10}} = \frac{5.1}{0.547} = 9.32$

df = 9, critical t (α = 0.05, one-tailed) = 1.833

Decision: Since 9.32 > 1.833, reject H₀

Conclusion: Strong evidence that training improves typing speed.

Why Paired Tests Are More Powerful

Two-Proportion z-Test

Compares proportions between two independent groups.

Two-Proportion z-Test

$z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$

Where $\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}$ (pooled proportion)

Two-Proportion Test

Question: Is there a difference in cure rates between two drugs?

Drug A: 80 cured out of 100 (p̂₁ = 0.80) Drug B: 65 cured out of 100 (p̂₂ = 0.65) α = 0.05, two-tailed

Hypotheses:

H₀: p₁ = p₂
H₁: p₁ ≠ p₂

Pooled proportion: $\hat{p} = \frac{80 + 65}{100 + 100} = \frac{145}{200} = 0.725$

Standard error: $SE = \sqrt{0.725(0.275)\left(\frac{1}{100} + \frac{1}{100}\right)} = \sqrt{0.199 \times 0.02} = 0.0631$

Test statistic: $z = \frac{0.80 - 0.65}{0.0631} = \frac{0.15}{0.0631} = 2.38$

Critical values: ±1.96

Decision: Since 2.38 > 1.96, reject H₀

Conclusion: Evidence suggests cure rates differ between drugs.

Independent vs Paired: How to Decide

Feature	Independent	Paired
Subjects	Different people in each group	Same people (or matched pairs)
Sample sizes	Can be different	Must be equal (n pairs)
Design	Random assignment to groups	Before/after, matching
Analysis	Compare two group means	Analyze differences

Identifying the Design

Independent samples:

Compare men’s and women’s salaries
Treatment group vs control group (different people)

Paired samples:

Weight before and after diet (same people)
Left hand vs right hand reaction time (same people)
Ratings of two products by same consumers

Assumptions and Conditions

For Independent t-Test

Random samples
Independent groups
Normal populations OR large n (each group)
Equal variances (if using pooled test)

For Paired t-Test

Random sample of pairs
Paired observations
Differences are approximately normal OR large n

For Two-Proportion Test

Independent random samples
n₁p̂₁ ≥ 10, n₁(1-p̂₁) ≥ 10
n₂p̂₂ ≥ 10, n₂(1-p̂₂) ≥ 10

Summary

In this lesson, you learned:

Independent t-test: Compares means of two separate groups
Paired t-test: Compares means when subjects are measured twice (analyze differences)
Two-proportion z-test: Compares proportions between two groups
Paired tests control for individual differences and are often more powerful
Use Welch’s t-test (unpooled) when variances may differ
Always check assumptions before conducting tests

Practice Problems

1. Group 1: n = 15, x̄ = 85, s = 7 Group 2: n = 18, x̄ = 80, s = 9 Test if means differ at α = 0.05.

2. 12 patients’ blood pressure before and after medication: d̄ = -8.5 (decrease), s_d = 6.2 Test if medication reduces blood pressure (one-tailed, α = 0.05).

3. Survey: 45 of 150 men and 62 of 180 women support a policy. Test if proportions differ at α = 0.05.

4. A researcher wants to test if a new fertilizer increases crop yield. She has 20 fields, and applies the fertilizer to half of each field. Should she use independent or paired t-test? Why?

Click to see answers

1. Independent t-test

$SE = \sqrt{\frac{49}{15} + \frac{81}{18}} = \sqrt{3.27 + 4.5} = \sqrt{7.77} = 2.79$

$t = \frac{85 - 80}{2.79} = 1.79$

df (conservative) = min(14, 17) = 14 Critical t = ±2.145

Since |1.79| < 2.145, fail to reject H₀ No significant difference between groups.

2. Paired t-test

H₀: μ_d = 0, H₁: μ_d < 0

$t = \frac{-8.5}{6.2/\sqrt{12}} = \frac{-8.5}{1.79} = -4.75$

df = 11, critical t (one-tailed, α = 0.05) = -1.796

Since -4.75 < -1.796, reject H₀ Strong evidence medication reduces blood pressure.

3. Two-proportion z-test

p̂₁ = 45/150 = 0.30, p̂₂ = 62/180 = 0.344 p̂ = (45 + 62)/(150 + 180) = 107/330 = 0.324

$SE = \sqrt{0.324(0.676)(\frac{1}{150} + \frac{1}{180})} = \sqrt{0.219 \times 0.0122} = 0.0517$

$z = \frac{0.30 - 0.344}{0.0517} = -0.85$

Critical z = ±1.96

Since |-0.85| < 1.96, fail to reject H₀ No significant difference in proportions.

4. Paired t-test

Each field serves as its own control. Half gets fertilizer, half doesn’t. The two measurements (treated vs untreated) come from the same field.

Pairing controls for field-to-field variation (soil quality, drainage, etc.), making it easier to detect the fertilizer effect.

Next Steps

Continue with more advanced testing:

ANOVA - Compare more than two groups
Chi-Square Tests - Categorical data
T-Test Calculator - Practice calculations

Person	Before	After	Difference (d)
1	45	52	7
2	38	41	3
3	55	60	5
4	42	49	7
5	50	53	3
6	35	42	7
7	48	51	3
8	41	47	6
9	52	58	6
10	44	48	4

Person	Before	After	Difference (d)
1	45	52	7
2	38	41	3
3	55	60	5
4	42	49	7
5	50	53	3
6	35	42	7
7	48	51	3
8	41	47	6
9	52	58	6
10	44	48	4

Comparing Two Groups

Independent Two-Sample t-Test

Degrees of Freedom

Pooled vs Unpooled Variance

Paired t-Test

Why Paired Tests Are More Powerful

Two-Proportion z-Test

Independent vs Paired: How to Decide

Assumptions and Conditions

For Independent t-Test

For Paired t-Test

For Two-Proportion Test

Summary

Practice Problems

Next Steps

Was this lesson helpful?

Person	Before	After	Difference (d)
1	45	52	7
2	38	41	3
3	55	60	5
4	42	49	7
5	50	53	3
6	35	42	7
7	48	51	3
8	41	47	6
9	52	58	6
10	44	48	4