Hours studied vs quiz score
The two knobs
The exact best line is score = 5.22 × hours + 39.82. It still misses, because real scores are noisy.
Loss if you only turned one knob
Each curve is a valley, and the dashed line is where you are. Training means walking to the bottom of the valley. The next lesson does that automatically.
Why squared error?
Every miss counts as positive. Missing by +10 and −10 shouldn't cancel out to "perfect".
Big misses count much more. One miss of 20 costs 400, but four misses of 5 cost only 100. Turn on "Draw the squares" and the big red squares show you which students the line is failing hardest.
It's a smooth valley. Look at the MSE curve: a smooth bowl with one bottom. That smoothness is what lets gradient descent slide down it. MAE's valley has a sharp point at the bottom, which is harder to slide into.
LLMs use a different loss (cross-entropy, lesson 5 onward) because they predict a word, not a number. But the idea is identical: one number for "how wrong", and every knob gets turned to make it smaller.
Every residual, one by one
| Hours | Real score | Line says | Miss | Miss² |
|---|---|---|---|---|
| 1.0 | 45.1 | 45.2 | −0.1 | 0 |
| 3.3 | 62.8 | 57.2 | 5.6 | 32 |
| 7.1 | 82.4 | 76.9 | 5.5 | 30 |
| 3.2 | 54.1 | 56.6 | −2.5 | 6 |
| 5.0 | 75.9 | 66.0 | 9.9 | 98 |
| 4.2 | 69.5 | 61.8 | 7.7 | 59 |
| 5.0 | 61.1 | 66.0 | −4.9 | 24 |
| 8.5 | 80.3 | 84.2 | −3.9 | 15 |
| 3.9 | 65.8 | 60.3 | 5.5 | 30 |
| 6.6 | 67.9 | 74.3 | −6.4 | 41 |
| 8.9 | 85.4 | 86.3 | −0.9 | 1 |
| 3.6 | 63.6 | 58.7 | 4.9 | 24 |
| 9.1 | 95.0 | 87.3 | 7.7 | 59 |
| 7.4 | 76.3 | 78.5 | −2.2 | 5 |
| 9.2 | 86.3 | 87.8 | −1.5 | 2 |
| 1.9 | 49.5 | 49.9 | −0.4 | 0 |
| 8.8 | 85.2 | 85.8 | −0.6 | 0 |
| 4.5 | 56.3 | 63.4 | −7.1 | 50 |
| 7.4 | 72.3 | 78.5 | −6.2 | 38 |
| 2.1 | 54.9 | 50.9 | 4.0 | 16 |
| 2.2 | 42.2 | 51.4 | −9.2 | 85 |
| 6.1 | 77.8 | 71.7 | 6.1 | 37 |
| 4.8 | 57.2 | 65.0 | −7.8 | 60 |
| 2.9 | 49.7 | 55.1 | −5.4 | 29 |
| Average of the last column = MSE | 31.0 | |||
The code that just ran
Everything on this page came from calling this code, which lives at the top of this same file.
Show or hide the source
final class Line
{
public function __construct(public float $m, public float $b) {}
// The model itself. That's it: one multiplication and one addition.
public function predict(float $x): float
{
return $this->m * $x + $this->b;
}
// Residual = how far each real point is from the line (real − predicted).
public function residuals(array $points): array
{
return array_map(fn($p) => $p['y'] - $this->predict($p['x']), $points);
}
// MEAN SQUARED ERROR: square every miss, then average.
// Squaring makes every miss positive and punishes big misses much more.
public function mse(array $points): float
{
$sum = 0.0;
foreach ($this->residuals($points) as $r) $sum += $r * $r;
return $sum / count($points);
}
// MEAN ABSOLUTE ERROR: the plain average size of a miss.
public function mae(array $points): float
{
$sum = 0.0;
foreach ($this->residuals($points) as $r) $sum += abs($r);
return $sum / count($points);
}
// For a straight line there is an exact formula for the lowest-MSE line
// ("least squares"). Most models have no such formula, which is why the
// next lesson, gradient descent, exists.
public static function bestFit(array $points): self
{
$n = count($points);
$mx = array_sum(array_column($points, 'x')) / $n;
$my = array_sum(array_column($points, 'y')) / $n;
$num = 0.0; $den = 0.0;
foreach ($points as $p) {
$num += ($p['x'] - $mx) * ($p['y'] - $my);
$den += ($p['x'] - $mx) ** 2;
}
$m = $den ? $num / $den : 0.0;
return new self($m, $my - $m * $mx);
}
}