Bivariate Data & Correlation

Fit a line, read its slope in context, and judge how well it fits — then resist the urge to claim a cause.

Two students fit a line to the same scatter plot. Their lines are slightly different. Who is right?

Both might be. But you can measure which line fits better, and the measurement is simple.

the line summarises; the gaps tell you how well a gap above the line is a positive residual, below it a negative one

Drop a little line from each dot to your fitted line. That gap is how much the line missed that dot by — its residual.

Above the line is a positive residual, below is negative, and for a least-squares line they cancel out to zero.

Now plot the residuals on their own. If the line was right, they should look like a formless jumble.

If instead they make an arch — negative at both ends, positive in the middle — the data curves and a line was the wrong shape. This is the one place in mathematics where finding no pattern is the good outcome.

A line of best fit through a scatter plot the line of best fit runs through the middle of the cloud hours practised score roughly half above, half below use it to predict inside the data, not far outside it a trend is not a cause — both may be following something else entirely

Reading r

r close to 1: strong positive linear relationship. Close to -1: strong negative. Close to 0: little or no linear relationship.

r = 0.9 is strong; r = 0.3 is weak. The sign matches the slope's sign, always.

r measures only how linear the pattern is. A perfect U-shaped curve can have r near zero.

Sign then size

Read the sign for direction, then the size for strength. r = -0.95 is a very strong relationship, not a weak one.

Residuals

Residual = actual - predicted. A positive residual means the real value was above the line.

Add up the residuals for a least-squares line and you get zero — the line splits the misses evenly.

A small residual means the line predicted that case well.

Actual minus predicted

Always in that order. If the real score was 82 and the line predicted 78, the residual is +4.

The residual plot

Plot each residual against its x value. If a line was the right model, the residuals look like random scatter around zero.

A clear curve or arch in the residual plot means the true relationship is not linear, even if r looked respectable.

This is the honest check, and it catches bad models that r alone will let through.

Pattern is bad news

In a residual plot, structure means the model is wrong. You want a formless cloud — the one place in mathematics where no pattern is the good outcome.

Correlation is still not causation

A strong r says the two variables moved together. It says nothing about why.

Three explanations always remain: x causes y, y causes x, or something else causes both.

Only a randomised experiment can rule out the third. Observational data never can.

Worked examples

Interpreting r

A study finds r = -0.87 between hours of television and test score. Describe the relationship.

  1. Negative sign: more television goes with lower scores. The direction.
  2. |-0.87| is close to 1, so it is strong. The strength.
  3. So: a strong negative linear association. Both pieces reported.
  4. But television may not be the cause — a third factor could drive both. Correlation is not causation.

A strong negative linear association; no cause established

A residual

A regression line predicts y = 3x + 10. For x = 5 the actual value was 28. Find the residual.

  1. Predicted: 3(5) + 10 = 25 Substitute into the line.
  2. Residual = actual - predicted = 28 - 25 Always in that order.
  3. = +3 Positive, so the actual value sat above the line.

+3

Judging a model

A residual plot shows residuals that are negative at both ends and positive in the middle. What does that tell you?

  1. There is a clear pattern — an arch. Residuals should look random.
  2. The data curves, but the line cannot. The line under-predicts in the middle and over-predicts at the ends.
  3. So a linear model is the wrong choice here. A quadratic or exponential model would fit better.

The relationship is not linear — try a different model

Words to know

Regression line
The line of best fit computed by a formula rather than by eye.
Correlation coefficient
r, between -1 and 1. Its sign gives the direction, its size the strength.
Residual
Actual value minus predicted value.
Residual plot
A plot of residuals against x, used to judge whether a line was the right model.

Mistakes to watch for

Calling r = -0.9 weak because it is negative. Read the sign for direction and the size for strength. -0.9 is nearly as strong as a relationship gets.

Computing predicted minus actual for a residual. It is always actual minus predicted, in that order, so that a point above the line gives a positive residual. The other order reverses every sign.

Treating a high r as proof of cause. Three explanations always survive: x causes y, y causes x, or something else causes both. Only a randomised experiment can rule out the third.

Ignoring a curved residual plot because r looked respectable. r measures how linear the pattern is; the residual plot shows whether a line was the right model at all. The residual plot is the honest one.

The correlation coefficient r was developed by Karl Pearson in the 1890s, building on Galton's work.

Its sign and size are all it reports: r = -0.95 is a very strong relationship, not a weak one, because strength lives in the size and direction in the sign.

r also only sees straight lines. A perfect U-shaped relationship — every point exactly on a parabola — can give r of almost exactly zero.

Browse all courses