One function, two readings: the density of the data and the likelihood of the weight

Chapter 4 · One input x, one weight w, noise variance σ². The model says the target t is Gaussian around the prediction w·x: p(t | x, w) = N(w·x, σ²). This is a single function of two variables, t and w. Drawn as a heat map, a vertical slice (w fixed) gives a density over t, and a horizontal slice (t fixed) gives the likelihood over w. Same numbers, two readings.

The function of both variables

Darker means a higher value of p(t | x, w). The dashed line is the prediction t = w·x. Blue vertical line: w fixed, varying t. Red horizontal line: t fixed, varying w. The dot is where the slices cross.

As a density over the data t (w fixed)

A genuine density over t: it integrates to 1. Its centre is w·x and its spread does not depend on t.

As the likelihood over the weight w (t fixed)

The likelihood over w: not a density, and its area is not 1. It is a Gaussian in w centred at t/x.

Height at the dot, p(t | x, w)
Area under the left curve (over t)
Area under the right curve (over w)
Right curve: centre t/x
Right curve: precision x²/σ²
The height at the dot is one number, p(t | x, w), and it is the same in both slices. What differs is what you are allowed to say about the curve. Over t it is a probability density, so the area is 1. Over w it is the likelihood, which only tells you how well each w explains the fixed observation. Its area is 1/|x|, and its precision x²/σ² is the quantity that appears in the posterior as XᵀX/σ². Strictly, the word likelihood names only the second reading, the function of w with t fixed. The same expression read as a function of t is the model's density for the data. The likelihood of w is highest at w = t/x, where the prediction w·x hits the observed t exactly.