www.john-weber.com

Chapter 5: Regression

Knowledge Prerequisites

Scatterplots

declarative knowledge (definitions)
1. independent variable
2. dependent variable
3. response variable
4. explanatory variable
5. correlation
procedural knowledge
1. how to construct a scatteplot
conditional knowledge
1. how to describe a scatterplor
resources
1. Chapter 4 of text.

Slope-intercept equation of line

declarative knowledge (definitions)
1. know what $m$ and $b$ represent in $y = m x + b$
procedural knowledge
1. how to graph a line
conditional knowledge
1. know how $m$ and $b$ affect the graph of $y = m x + b$
resources

Learning Goals

declarative knowledge (definitions)
1. regression line
2. least-squares regression line, $\hat{y} = a + b x$
3. influential observations
4. lurking variables
5. extrapolation
procedural knowledge
1. how to calculate the least-squares regression line
2. how to make a prediction using the least-squares regression line
conditional knowledge
1. know the difference between an outlier and an influential observation.
2. be aware of the potential problems of predictions using the least-squares regression line.
3. know the facts about least-squares regression (see pp. 110-111 of text).
4. know that "association does not imply causation".
5. know what $r^{2}$ tells about the relationship.

Questions to Ponder

What is the best way to study/prepare for the concepts in this section? Are there are other methods to study these concepts?
What questions do you need to ask the instructor?

Purpose of this Section

To quantify the relationship between two variables (i.e., to construct a mathematical model for a linear relationship between two variables).

General Notes

The least $-$ squares regression line

We need a method for constructing a regression line that will be consistent for all looking at the data. Here is a good website (http://www.ruf.rice.edu/~lane/stat_sim/reg_by_eye/index.html) that shows how close your guess at the regression line is to the line constructed by the method of least squares. This is an excellent way to develop a feel for linear regression!

The leastsquares regression line of $y$ on $x$ is the line that makes the sum of the squares of the deviations of the data points from the line in the vertical direction as small as possible. The least-squares regression line is the line $\hat{y} = a + b x$ where $\hat{y}$ (called $y$ -hat) is the predicted value for the response variable. Because of the scatter of points about the line, the predicted response will usually not be exactly the same as the actually observed response $y$ .

Here are steps to find the least $-$ squares regression line using the TI-83 calculator.

Residuals

A residual is the difference between an observed value of the response variable and the value predicted by the regression line:

residual = y - \hat{y}

There are residuals for each data point. Residuals are positive when the data point is above the regression line and residuals are negative when the data point is below the regression line.

Because the residuals show how far the data fall from our regression line, examining the residuals helps assess how well the line describes the data. The mean of the residuals from the leasts-quares regression line is always zero. A residual plot is a scatter plot of the regression residuals against the explanatory variable. Residual plots also help us ass the fit of a regression line.

Here are steps to finding and graphing the residuals using the TI-83 calculator.

Here are some key things to consider when examining the residual plot (see pp. 118121 of your text for example graphs of each of the key ideas below):

Ideal case: Random pattern around 0, no unusual individual observations.
Curved pattern: Relationship is NOT linear.
Increasing (or decreasing) spread as $x$ increases: Prediction is less accurate when $x$ is large (small).
Individual points with large residuals: These points are possible outliers.
Individual points far from others in the $x$ -direction: These are called influential observations.

Resources

Visual Statistics - http://www.seeingstatistics.com/seeingTour/intro/why3.html - This applet graphs some common geometric statistical concepts: data points (black dots), residuals (red lines), error squares (yellow squares), and regression or model line (blue line). Use your mouse to drag the blue line and see how the geometric objects change.
Regression by Eye - http://www.ruf.rice.edu/~lane/stat_sim/reg_by_eye/index.html - This applet gives you a scatterplot and you can use the mouse to indicate what seems to be the best straight line approximating the data. This is an excellent way to develop a feel for linear regression!
Correlation and Regression Calculator - http://calculators.stat.ucla.edu/correlation.php - This calculator returns the means, variances, regression coefficients, covariance, correlation coefficient, plus a scatterplot with the two regression lines.

Back to John Weber's MATH 1431 Page

Back to john-weber.com