Figure 5.2: fitting a Gaussian to each class

Each class c gets its own Gaussian N(μ_c, Σ_c) fitted from the points of that class only, with the maximum-likelihood estimates of Equations 5.4 and 5.5: μ_c is the mean of the class's points and Σ_c = (1/N_c) Σ (x − μ_c)(x − μ_c)^T. The contours are curves of equal density, which for a Gaussian are ellipses. The data are synthetic (30 points per class, as in the book) and are not the book's own points. Drag a point and watch its class's fit change.

Click empty space to add a point to: class 1 class 2 class 3 (double-click a point to delete it)

1 sd contour 2 sd contour 3 sd contour shade the region where each class has the highest posterior force Σ = I (as in Exercise 5.1)

Fitted parameters

Where does x_new belong? (drag the black cross)

Columns as in Table 5.1: density of x_new under each class, the prior N_c/N, their product, and the normalised posterior probability.

What to try: (1) Drag one class-3 point far away: the covariance of that class, and so the shape of its contours, changes a lot, because the covariance is built from squared distances. (2) Class 3 is long and thin, so its density falls off very quickly across the thin direction, and a point just beside it gets a tiny density. (3) Turn on Σ = I: every contour becomes a circle around the class mean and the regions are separated by straight lines. (4) Delete points until a class has two or fewer: Σ_c becomes singular and the fit is unreliable, which is where the Bayesian treatment of Exercises 5.1 and 5.2 helps.