Likelihood of a Gaussian linear model

Chapter 2 · Model: t_n = w₀ + w₁(x_n − 1900) + ε_n, with noise ε_n ~ N(0, σ²). So for each year the model says the winning time follows a Gaussian centred on the line. The likelihood of a data point is the height of that Gaussian at the time that was actually observed.

Event Show the Gaussian at every year

━ maximum-likelihood line ● chosen data point ● other data

Data point

Noise σ

The same Gaussian drawn on its own. The red dashed line marks the observed time; its height is the likelihood of this data point.

Predicted time: s

Observed time: s

Likelihood p(t_n | x_n, w, σ²):

log-likelihood of all data:

The Gaussian density is p(t | x, w, σ²) = (2πσ²)^−1/2 exp( −(t − w^Tx)² / (2σ²) ). The fitted line uses the least-squares w, which is also the maximum-likelihood w for this model, and the fitted noise level is σ² = (1/N) Σ (t_n − w^Tx_n)². Make σ smaller or larger than the fitted value and watch the log-likelihood of all the data fall.