PapaMarks
Library › Linear Regression
Further Mathematics

Linear Regression

Revise Linear Regression for Further Mathematics — revision notes and instant AI marking. Free to start.

📖 Revision notes · preview
Edexcel Further Statistics 2

Linear Regression

The Big Idea When two things are linked, you can draw one "line of best fit" through the data (using a formula, not a ruler) that lets you predict one value from the other — and then check with numbers, not just a gut feeling, whether that line is actually any good.

Summary — What This Chapter Covers

  • Least squares regression finds the line y = a + bx that minimises the total squared distance between the line and every data point.
  • Sxx, Sxy, Syy are three "helper sums" you calculate first — everything else (the gradient, the intercept, how good the fit is) is built from them.
  • Interpolation vs extrapolation — predicting inside your data range is safe; predicting outside it is a guess dressed up as maths.
  • Coded data lets you work with small, friendly numbers instead of huge ones, then convert your answer back using substitution.
  • Residuals are the leftover gaps between what actually happened and what your line predicted — plotting them tells you if a straight line was ever the right shape to use.
  • RSS (Residual Sum of Squares) turns "how good is my fit" into a single number — smaller is better.

1. Least Squares Linear Regression

Imagine you've got a scatter of points on a graph — say, temperature against reaction time. You could draw a line through them freehand, but everyone would draw it slightly differently. Least squares regression removes the guesswork: it's the one specific line that makes the total "error" between the line and the real points as small as mathematically possible.

Here's the trick to understanding *why* it's called "least squares": for every point, there's a vertical gap between where the point actually is and where the line says it should be. That gap is called a residual. Some residuals are positive (point is above the line), some are negative (point is below). If you just added them up, positives and negatives would cancel out and hide how good or bad the fit really is. So instead, you square every residual first (which also makes them all positive) and add those up. The regression line is the one specific line that makes this sum — the sum of squared residuals — as small as it can possibly be. That's the whole idea behind the name "least squares."

The Regression Line
y = a + bx
where y is the dependent variable (the thing being predicted), x is the independent variable (the thing you already know), b is the gradient of the line, and a is the y-intercept.
The Three Helper Sums
Sxx = Σx² − (Σx)²/n
Sxy = Σxy − (Σx)(Σy)/n
Syy = Σy² − (Σy)²/n
These aren't random — they're standardised measures of "spread" and "shared spread." Sxx measures how spread out the x-values are. Syy measures how spread out the y-values are. Sxy measures how x and y vary together — this is what tells you the relationship exists at all.
Gradient and Intercept
b = Sxy / Sxx      a = ȳ − bx̄
You always find b first (it only needs the helper sums), then use it to find a. Note that ȳ = Σy/n and x̄ = Σx/n are just the means of the data — and a neat fact worth remembering: the regression line always passes through the point (x̄, ȳ), the "centre of mass" of your data. That's exactly why the formula for a works — you're finding the intercept that forces the line through that centre point.

Worked Example

A researcher records temperature (x, °C) against the time taken for a reaction to finish (y, seconds):

Temp (°C)22.023.024.025.026.027.028.029.0
Time (s)43.241.639.938.136.533.631.730.1

Given: n = 8, Σx = 204, Σx² = 5244, Σy = 294.7, Σy² = 11012.53, Σxy = 7434.

Step-by-step:
Sxx = 5244 − 204²/8 = 42
Sxy = 7434 − (204 × 294.7)/8 = −80.85
b = −80.85 / 42 = −1.925
x̄ = 204/8 = 25.5,   ȳ = 294.7/8 = 36.8375
a = 36.8375 − (−1.925 × 25.5) = 85.925
So the regression line is: y = 85.9 − 1.93x (3 s.f.)
Notice b came out negative here. That's completely fine — it just means as x increases, y decreases (hotter temperature → faster/shorter reaction time). Don't panic and assume you've made an error just because the gradient isn't positive. Always sanity-check the sign against what the context suggests.
Practice Question 1

Using Sxx = 42 and Sxy = −80.85 from above, if instead Σy = 320 and n = 8, what would ȳ be, and how would that change the value of a (keeping b = −1.925 and x̄ = 25.5)?

Interpreting a and b in Context

This is the part students lose easy marks on — examiners always ask you to explain what a and b mean in real-world terms, not just state the numbers.

  • a (the intercept) is the predicted value of y when x = 0. In context: "When the temperature is 0°C, the predicted time for the reaction to complete is 85.9 seconds."
  • b (the gradient) is how much y changes for every 1-unit increase in x. In context: "For every 1°C increase in temperature, the time taken for the reaction decreases by 1.93 seconds."
The examiner wants THREE things in your interpretation: (1) the correct direction (increase/decrease), (2) the correct units, and (3) it phrased about the actual variables in the question — not "x" and "y". Always name what x and y represent.

2. Interpolation & Extrapolation

Once you have your regression equation, you can plug in an x-value to predict y. But whether you should trust that prediction depends entirely on where that x-value sits relative to your original data.

  • Interpolation — predicting a y-value for an x that falls within the range of x-values you actually collected. This is reliable, because your line was built from evidence covering that exact region.
  • Extrapolation — predicting for an x outside that range. This is unreliable, because you have no evidence the same straight-line relationship still holds out there. The pattern could curve, level off, or reverse entirely beyond your data.
Think of it like a road you've only driven the middle section of. Interpolation is filling in a pothole you can see clearly ahead. Extrapolation is guessing what the road looks like past the horizon — it might keep going straight, or it might suddenly turn into a cliff. You just don't know.

Worked Example (continued)

Using y = 85.9 − 1.93x, with the original data spanning x = 22.0 to 29.0:

At x = 25.5 (inside range → interpolation): y = 85.9 − 1.93(25.5) = 36.7s. Reliable — 25.5 sits comfortably between 22 and 29.

At x = 30.0 (outside range → extrapolation): y = 85.9 − 1.93(30) = 28s. Unreliable — 30°C is beyond the highest recorded temperature (29°C), so we're assuming the relationship keeps behaving the same way, which we can't confirm.
Practice Question 2

A data set of x-values ranges from 10 to 50. Using a regression equation, would predicting y at x = 45 be interpolation or extrapolation? What about x = 55? Explain your reasoning for both.

3. Coded Data

Sometimes the real data involves huge or awkward numbers (think: national income in the billions, or measurements in nanometres). To make the arithmetic manageable, the question will "code" the data using a substitution formula — you work entirely in the coded world, then translate your final answer back using algebra.

The golden rule: coded data problems are just substitution problems. Whatever the coding formula says x and y equal in terms of the coded letters, sub that straight into your regression equation and rearrange. Don't try to shortcut it — write out every substitution step, because that's where marks are awarded.

Worked Example

Data is coded using x = 3 − s and y = −16t. The regression line for y on x is found to be y = 3x + 8.

Part (i): Find t when s = 216.

First find x: x = 3 − 216 = −216.
Substitute into the regression line: y = 3(−216) + 8 = −640.
Now use the coding formula for y to isolate t: −640 = −16t  ⟹  t = 40.

Part (ii): Find the regression line of t on s directly.

Substitute the coding formulas x = 3 − s and y = −16t straight into y = 3x + 8:
−16t = 3(3 − s) + 8
−16t = 9 − 3s + 8 = 17 − 3s
t = (−17 + 3s) / 16 = 1.125s − 6.375 ... wait, watch the signs carefully here — this is exactly the step where students lose marks by mis-distributing negatives. Divide every term by −16 individually and double check the sign of each.
Practice Question 3

Data is coded using u = x − 100 and v = y/10. The regression line of v on u is v = 2u + 5. Find the regression line of y on x.

4. Residuals

A residual measures exactly how wrong the regression line was for one specific point. It's the difference between what actually happened (the observed y-value) and what the line predicted (the fitted y-value).

Residual Formula
residual = yi − (a + bxi)
where (xi, yi) is an actual data point, and (a + bxi) is the value the regression line predicts at that same xi. A key fact: the residuals always sum to exactly 0 across the whole data set — this comes directly from how least squares regression is defined, and it's a handy way to check your working.

Residuals are useful for more than just checking arithmetic — they let you test whether a straight line was even the right shape of model to use in the first place. Plot each residual against its x-value (or fitted value):

  • Random scatter around zero, no pattern → the linear model is suitable. This tells you the leftover "error" isn't systematic — it's just noise.
  • A clear pattern (like a curve, or residuals that grow/shrink in a trend) → the linear model is not suitable. A pattern in the residuals means there's structure in the data that a straight line failed to capture — often it means the true relationship is curved.
Think of residuals as "what the line got wrong." If those mistakes are totally random, your line has captured everything it can — the leftovers are just noise. But if the mistakes themselves follow a pattern (e.g. the line under-predicts at low x, over-predicts in the middle, under-predicts again at high x — a wave shape), that pattern is information your straight line missed. It's a sign you probably need a curve instead.
Practice Question 4

A regression line gives y = 12 + 4x. For the data point (5, 34), calculate the residual. What does the sign of your answer tell you about where the point lies relative to the line?

5. Residual Sum of Squares (RSS)

Looking at individual residuals or a scatter plot is useful for spotting patterns, but sometimes you need a single number that summarises "how good is this fit overall" — especially when comparing two different data sets or two different models. That number is RSS.

RSS Formula
RSS = Syy − (Sxy)² / Sxx
This is literally the sum of every squared residual added together — but this formula lets you calculate it directly from your helper sums, without having to compute and square every individual residual by hand. Smaller RSS = better fit (the line sits closer to the data overall).

Worked Example (continued)

Using the reaction-time data: Sxy = −80.85, Sxx = 42, and Syy = 11012.53 − 294.7²/8 = 156.51875.

RSS = 156.51875 − (−80.85)²/42 = 156.51875 − 155.63625 = 0.8825

A second data set has RSS = 0.43. Since 0.43 < 0.8825, the second data set is more suitable for a linear model — its smaller RSS shows the data points sit closer to their regression line, meaning less unexplained "leftover" variation.

Don't compare RSS values from data sets with wildly different scales or numbers of points and assume that alone tells the full story — RSS is only a fair comparison when the two data sets are genuinely on comparable footing (similar y-scale, similar n). Within an exam question though, you can safely trust "smaller RSS = better fit" when directly asked to compare two RSS values.
Practice Question 5

For a data set, Sxx = 60, Sxy = 45, and Syy = 40. Calculate the RSS.

What to Memorise

Sxx, Sxy, Syy

Sxx = Σx² − (Σx)²/n  |  Sxy = Σxy − ΣxΣy/n  |  Syy = Σy² − (Σy)²/n. Given in the formula booklet — know how to use them, not recite them from memory.

Regression Line

y = a + bx, where b = Sxy/Sxx and a = ȳ − bx̄. Always find b first.

Interpolation vs Extrapolation

Interpolation = inside the data range = reliable. Extrapolation = outside the range = unreliable.

Coded Data

Substitute the coding formulas directly into the regression equation, then rearrange carefully.

Residual

residual = yi − (a + bxi). All residuals in a data set sum to exactly 0.

RSS

RSS = Syy − (Sxy)²/Sxx. Smaller RSS always means a better-fitting linear model.

Concepts Checklist

Exam Tips — Common Mistakes & Mark-Scheme Traps

Mistake 1: Finding a before b. You can't — the formula for a needs b already calculated. Always find Sxx and Sxy → b → x̄ and ȳ → a, in that exact order.
Mistake 2: Giving a bare numerical prediction without commenting on reliability. If a question gives you the range of the original data and then asks you to predict a value, there is almost always a mark specifically for stating whether it's interpolation or extrapolation.
Mistake 3: Interpreting a or b using "x" and "y" instead of the real-world variables. "For every increase in x, y decreases by b" scores no marks — you must name the actual quantities from the question, e.g. "for every 1°C increase in temperature..."
Mistake 4: Sign errors when rearranging coded regression equations. When a coding formula involves a minus sign (e.g. x = 3 − s), that negative has to be carried through every subsequent step — this is the single most common place marks are lost in coded-data questions. Write out each line of algebra rather than skipping steps.
Mistake 5: Confusing "large residual" with "bad model." One or two large residuals might just mean one or two outliers — it's the overall pattern across all residuals (via a residual plot) or the overall RSS that tells you if the model itself is unsuitable.
What examiners actually look for: full method shown (not just a final answer — Sxx, Sxy, and b/a should all appear as visible steps), correct rounding only at the very final answer, and a genuine sentence of interpretation using the context of the question wherever asked. Half-explained context answers ("a is the y-intercept") don't score interpretation marks — the examiner wants what that intercept means for the real scenario.
Linear Regression Revision Guide · Edexcel Further Statistics 2 · Built for offline study
Also in the full note
  • 2. Interpolation & Extrapolation
  • Exam Tips — Common Mistakes & Mark-Scheme Traps
What's inside
📖 Revision notes ✦ AI flashcards ✓ Instant AI marking

Read the full Linear Regression notes free

That's the preview — create a free account to read the rest, plus flashcards and practice questions with instant AI marking. No credit card.

Unlock the full notes free →