Sampling Distributions

What is a Sampling Distribution?

A sampling distribution is the probability distribution of a statistic (like the mean or proportion) obtained from all possible samples of a specific size from a population.

Think of it this way:

1. Take a random sample from a population
2. Calculate a statistic (e.g., the mean)
3. Repeat this process many, many times
4. The distribution of all those statistics is the sampling distribution

Why Sampling Distributions Matter

Sampling distributions are fundamental to inferential statistics because they:

• Allow us to make inferences about populations from samples
• Help us understand sampling variability
• Enable us to calculate probabilities and confidence intervals
• Form the basis for hypothesis testing

Without understanding sampling distributions, we can’t properly interpret statistical tests or confidence intervals.

Population vs. Sample vs. Sampling Distribution

Let’s clarify three important distributions:

Distribution	Description	Example
Population Distribution	Distribution of all values in the population	Heights of all adults in the US
Sample Distribution	Distribution of values in one sample	Heights of 100 randomly selected adults
Sampling Distribution	Distribution of a statistic from all possible samples	Distribution of mean heights from all possible samples of size 100

Notation

Symbol	Meaning	Context
$\mu$	Population mean	Parameter (unknown)
$\sigma$	Population standard deviation	Parameter (unknown)
$\bar{x}$	Sample mean	Statistic (calculated)
$s$	Sample standard deviation	Statistic (calculated)
$\mu_{\bar{x}}$	Mean of sampling distribution	Theoretical value
$\sigma_{\bar{x}}$	Standard deviation of sampling distribution (standard error)	Theoretical value

The Central Limit Theorem (CLT)

The Central Limit Theorem is one of the most important theorems in statistics.

Statement of the CLT

For a population with mean $\mu$ and standard deviation $\sigma$, the sampling distribution of the sample mean $\bar{x}$ based on samples of size $n$ has these properties:

1. Mean: $\mu_{\bar{x}} = \mu$
2. Standard Deviation (Standard Error): $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$
3. Shape: As $n$ increases, the distribution becomes approximately normal, regardless of the shape of the population distribution

This reads: “The sample mean is approximately normally distributed with mean μ and standard deviation σ/√n”

When Does the CLT Apply?

The CLT works under two conditions:

1. Large sample size: Generally, $n \geq 30$ is considered sufficient
2. Any population distribution: The population can be skewed, uniform, bimodal—it doesn’t matter!

Exception: If the population is already normal, the sampling distribution is exactly normal for any sample size.

Standard Error

The standard error (SE) is the standard deviation of the sampling distribution.

Key Points About Standard Error:

1. Measures sampling variability: How much sample means vary from sample to sample
2. Decreases with sample size: Larger samples give more precise estimates
3. Different from standard deviation: SD measures variability of individuals; SE measures variability of means

Practical Application: Calculating Probabilities

Once we know the sampling distribution is approximately normal, we can calculate probabilities using z-scores.

Effect of Sample Size

Sample size has a profound effect on the sampling distribution:

Small Samples (n small)

• Larger standard error
• More sampling variability
• Sample means spread out more
• Less reliable estimates

Large Samples (n large)

• Smaller standard error
• Less sampling variability
• Sample means cluster around μ
• More reliable estimates
• Closer to normal distribution (CLT)

Sampling Distribution of Proportions

The Central Limit Theorem also applies to proportions.

For a population proportion $p$:

Where:

• $\hat{p}$ = sample proportion
• $p$ = population proportion
• $n$ = sample size

Conditions for Normal Approximation:

• $np \geq 10$ and $n(1-p) \geq 10$

Relationship to Confidence Intervals

Sampling distributions form the foundation for confidence intervals:

• The standard error tells us how precise our estimate is
• The normal distribution tells us what range captures 95% of sample means
• This leads directly to: $\bar{x} \pm 1.96 \times SE$

Common Misconceptions

❌ Misconception 1: “Larger samples reduce population variability”

Reality: Sample size affects the standard error (precision of estimates), not the population standard deviation.

❌ Misconception 2: “The CLT makes the population normal”

Reality: The CLT makes the sampling distribution of the mean approximately normal, regardless of the population shape.

❌ Misconception 3: “We need to know the population mean”

Reality: In practice, we use sampling distributions to estimate the unknown population mean from sample data.

❌ Misconception 4: “Any sample size works”

Reality: While the CLT eventually works for any distribution, very skewed or heavy-tailed distributions may require larger samples.

Finite Population Correction

When sampling without replacement from a small population, we need a correction factor:

Where $N$ is the population size.

Rule of thumb: Use this correction when $n > 0.05N$ (sample is more than 5% of population).

Practical Implications

For Researchers:

1. Larger samples are better - But there are diminishing returns
2. Normal approximation is robust - Works even when populations aren’t normal
3. Estimate precision - Standard error quantifies uncertainty

For Sample Size Planning:

• To halve the standard error, quadruple the sample size
• To cut it by 75%, need 16 times the sample size
• Cost-benefit analysis is important

Summary

In this lesson, you learned:

• Sampling distributions describe the variability of statistics across samples
• The Central Limit Theorem states that sample means are approximately normal for large $n$
• Standard error measures the precision of sample estimates: $SE = \sigma/\sqrt{n}$
• Standard error decreases with sample size as $1/\sqrt{n}$
• Sampling distributions enable probability calculations and confidence intervals
• The CLT works for any population distribution with sufficient sample size

Next Steps

Apply your knowledge of sampling distributions:

• Confidence Intervals - Use SE to create ranges for parameters
• Hypothesis Testing - Use sampling distributions to test claims
• Z-Score Calculator - Practice probability calculations