The coin posterior and its Laplace approximation

Chapter 4, Exercise 4.3 · After y heads in N tosses, with a Beta(α, β) prior on the head probability r, the exact posterior is Beta(δ, γ) with δ = α + y and γ = β + N − y. The Laplace approximation is a Gaussian centred at the mode r̂ = (δ − 1)/(δ + γ − 2) with variance (δ − 1)(γ − 1)/(δ + γ − 2)³. Move the sliders and watch where the Gaussian is a good stand-in.

Examples:

Prior α

Prior β

Tosses N

Heads y

y-axis maximum Show the prior

━ exact posterior Beta(δ, γ) ╍ Laplace Gaussian shaded red: Gaussian mass that falls outside [0, 1] dotted: the mode r̂. The x-axis is fixed at −0.25 to 1.25 so you can see the Gaussian run past the ends.

Posterior parameters δ, γ

Mode r̂ (Laplace mean)

Laplace variance σ²

Exact mean

Exact variance

Gaussian mass outside [0, 1]

The Gaussian is centred on the peak of the posterior, so it is always right at the mode and wrong in the tails. With many counts the beta density becomes close to symmetric and the two curves agree. With few counts, or a mode near 0 or 1, the beta is skewed and bounded, and the Gaussian is not: its centre sits at the mode rather than the mean, and it spills past the ends. If δ or γ is at most 1, the posterior has no interior peak and the Laplace recipe cannot be used.