Simple Linear Regression

What is Linear Regression?

Linear regression finds the best straight line to describe the relationship between two variables and make predictions.

---

The Least Squares Method

The least squares method finds the line that minimizes the sum of squared errors (residuals).

---

Step-by-Step Example

---

Interpreting Coefficients

Slope (b₁)

Slope	Interpretation
Positive	Y increases as X increases
Negative	Y decreases as X increases
Near 0	Weak or no linear relationship

Units: The slope has units of Y per unit of X.

Intercept (b₀)

The predicted Y when X = 0. May or may not be meaningful depending on context.

---

Residuals and Model Fit

Properties of Residuals

• Sum of residuals = 0
• Mean of residuals = 0
• Residuals measure how far points are from the line

---

R-Squared (Coefficient of Determination)

Interpretation: R² is the proportion of variance in Y explained by X.

R²	Interpretation
0.0	Model explains nothing
0.5	Model explains 50% of variation
1.0	Model explains everything (perfect fit)

---

Standard Error of Estimate

Interpretation: Average distance of observations from the regression line.

---

Making Predictions

---

Inference for Regression

Testing the Slope

Confidence Interval for Slope

$b_1 \pm t_{\alpha/2} \times SE_{b_1}$

---

Assumptions of Linear Regression

Checking Assumptions

Assumption	How to Check
Linearity	Scatter plot, residual plot
Independence	Study design, residual patterns
Normality	Histogram or Q-Q plot of residuals
Equal variance	Residuals vs fitted values plot

---

Residual Plots

A residual plot (residuals vs. fitted values) should show:

• Random scatter around 0
• No patterns (curves, fans, clusters)

Good Residual Plot:        Bad (Curved):          Bad (Funnel):
      *                         *                      *
   *     *                    *   *                 *  *  *
*     *     *              *       *              *      *
   *     *                    *   *               *  *  *  *
      *                         *                        *

---

Summary

In this lesson, you learned:

• Linear regression predicts Y from X using $\hat{y} = b_0 + b_1 x$
• Least squares minimizes sum of squared residuals
• Slope (b₁): Change in Y for unit change in X
• Intercept (b₀): Predicted Y when X = 0
• R²: Proportion of variance explained (0 to 1)
• Residuals: Actual minus predicted values
• Assumptions: Linearity, independence, normality, equal variance
• Avoid extrapolation beyond data range

Practice Problems

1. A regression analysis yields: Price = 15,000 + 1,200(Age of Car) a) Interpret the slope b) Is the slope interpretation sensible? c) Predict price for a 5-year-old car

2. Given: r = 0.8, sx = 4, sy = 10, x̄ = 20, ȳ = 50 Find the regression equation for predicting Y from X.

3. A model has R² = 0.64. a) What correlation (r) produced this? b) What percentage of variance is unexplained?

4. Residual plot shows a curved pattern. What does this indicate?

Click to see answers

1. a) Slope = 1,200: Each additional year of age is associated with $1,200 increase in price b) **Not sensible!** Cars typically decrease in value with age. This might be antique/classic cars. c) Price = 15,000 + 1,200(5) = **$21,000**

2. b₁ = r(sy/sx) = 0.8(10/4) = 0.8(2.5) = 2.0 b₀ = ȳ - b₁x̄ = 50 - 2.0(20) = 50 - 40 = 10

Y = 10 + 2X

3. a) r = √0.64 = ±0.8 (need context to know sign) b) Unexplained = 1 - 0.64 = 0.36 or 36%

4. The linearity assumption is violated. The relationship is not linear. Consider:

• Transforming variables (log, square root)
• Using polynomial regression
• Using a different model type

Next Steps

Expand your regression knowledge:

• Multiple Regression - Multiple predictors
• Regression Diagnostics - Checking assumptions
• Standard Deviation Calculator - Calculate variability