Correlation Analysis

What is Correlation?

Correlation measures the strength and direction of the linear relationship between two quantitative variables. It answers the question: “When one variable changes, does the other tend to change in a predictable way?”

Correlation is a fundamental concept in statistics used across:

• Science (relationships between variables)
• Finance (stock price movements)
• Social sciences (behavior patterns)
• Medicine (risk factors and outcomes)

Visualizing Relationships

Before calculating correlation, always visualize your data with a scatter plot:

• X-axis: Independent variable (predictor)
• Y-axis: Dependent variable (outcome)
• Each point represents one observation

Scatter plots reveal:

• Direction of the relationship (positive or negative)
• Strength of the relationship (tight cluster vs. dispersed)
• Form of the relationship (linear vs. curved)
• Outliers that might affect the correlation

Pearson Correlation Coefficient

The Pearson correlation coefficient (denoted as $r$) is the most common measure of linear correlation.

Alternatively, using z-scores:

Where:

• $x_i, y_i$ = individual data points
• $\bar{x}, \bar{y}$ = means of x and y
• $z_x, z_y$ = standardized z-scores
• $n$ = number of observations

Properties of the Correlation Coefficient

Range

$-1 \leq r \leq +1$

Interpretation of r:

Value of r	Interpretation
$r = +1$	Perfect positive linear relationship
$0.7 \leq r < 1$	Strong positive correlation
$0.3 \leq r < 0.7$	Moderate positive correlation
$0 < r < 0.3$	Weak positive correlation
$r = 0$	No linear correlation
$-0.3 < r < 0$	Weak negative correlation
$-0.7 < r \leq -0.3$	Moderate negative correlation
$-1 < r \leq -0.7$	Strong negative correlation
$r = -1$	Perfect negative linear relationship

Key Properties:

1. Unitless: r has no units, making it comparable across different measurements
2. Symmetric: Correlation of X with Y equals correlation of Y with X
3. Linear only: r measures only linear relationships
4. Affected by outliers: Extreme values can greatly influence r

Positive vs. Negative Correlation

Positive Correlation ($r > 0$)

• As one variable increases, the other tends to increase
• Points slope upward from left to right
• Example: Study time and test scores

Negative Correlation ($r < 0$)

• As one variable increases, the other tends to decrease
• Points slope downward from left to right
• Example: Exercise frequency and resting heart rate

No Correlation ($r \approx 0$)

• No clear linear relationship
• Points appear randomly scattered
• Example: Shoe size and intelligence

Coefficient of Determination ($r^2$)

The coefficient of determination is the square of the correlation coefficient:

$r^2 = (\text{correlation coefficient})^2$

Interpretation

$r^2$ represents the proportion of variance in one variable that is predictable from the other variable.

Testing Correlation Significance

Is the observed correlation statistically significant, or could it have occurred by chance?

Hypotheses:

• $H_0: \rho = 0$ (no correlation in the population)
• $H_A: \rho \neq 0$ (correlation exists in the population)

Where $\rho$ (rho) is the population correlation coefficient.

Test Statistic:

With degrees of freedom: $df = n - 2$

Assumptions and Limitations

Assumptions for Pearson’s r:

1. Linearity: The relationship must be linear
2. Bivariate normality: Both variables should be approximately normally distributed
3. Homoscedasticity: Variance should be constant across the range
4. Independence: Observations should be independent
5. No extreme outliers: Outliers can dramatically affect r

Common Pitfalls

1. Correlation ≠ Causation

Just because two variables are correlated doesn’t mean one causes the other.

2. Outliers Can Mislead

A single outlier can create, destroy, or reverse a correlation.

3. Restricted Range

Correlations computed on a restricted range of values may underestimate the true relationship.

4. Lurking Variables

A third variable might be influencing both variables, creating a spurious correlation.

5. Nonlinear Relationships

Pearson’s r only measures linear relationships. Curved relationships won’t be captured.

Other Correlation Measures

Spearman’s Rank Correlation ($r_s$)

• Used for ordinal data or when data doesn’t meet assumptions
• Based on ranks rather than raw values
• More robust to outliers

Kendall’s Tau ($\tau$)

• Another rank-based correlation
• Preferred for small samples or many tied ranks

Point-Biserial Correlation

• Used when one variable is continuous and one is binary (0/1)

Practical Applications

1. Finance

• Correlation between stock prices
• Portfolio diversification

2. Medicine

• Relationship between risk factors and disease
• Dose-response relationships

3. Education

• Study habits and academic performance
• Test score validation

4. Psychology

• Personality traits and behavior
• Cognitive abilities

Summary

In this lesson, you learned:

• Correlation measures the strength and direction of linear relationships
• Pearson’s r ranges from -1 to +1
• $r^2$ represents the proportion of variance explained
• Always visualize data before computing correlations
• Correlation does not imply causation
• Watch out for outliers, non-linearity, and restricted ranges
• Statistical significance can be tested using a t-test

Next Steps

Build on your correlation knowledge:

• Simple Linear Regression - Use correlation to make predictions
• Correlation Calculator - Compute correlations quickly
• Scatter Plot Generator - Visualize relationships