A binary response dataset, with a Gaussian prior on the weights
Chapter 4 · Each object has two attributes and a label t. The model is P(t = 1 | x, w) = 1/(1 + exp(−wᵀx)) with prior w ~ N(0, σ²I). Draw weight vectors from the prior and see which classifiers they give, then compare with the weights that fit the data best (the MAP estimate). Data: logregdata.mat.
● t = 0 ■ t = 1 grey lines: boundaries wᵀx = 0 for the draws dark line: the latest draw ━ MAP fit
| w₁ | w₂ | log-likelihood | log prior | log g(w) | accuracy | |
|---|---|---|---|---|---|---|
| Latest prior draw | ||||||
| MAP fit |
log g(w) = log-likelihood + log prior: the log of the unnormalised posterior, which the MAP fit maximises. Accuracy classifies a point as t = 1 when P > 0.5.
A weight vector w defines the boundary wᵀx = 0, a line through the origin. Scaling w by a constant does not move this line, only how sharply the probability changes across it, so σ² controls the confidence of the draws but not their directions. A large σ² gives draws with extreme probabilities, most of which fit the data poorly. The MAP weights are found by Newton–Raphson on log g, which has a single maximum.