The gamma density and its Laplace approximation

Chapter 4 · The gamma density is p(y | α, β) = βα yα−1 e−βy / Γ(α) for y > 0. The Laplace approximation replaces it by a Gaussian centred at its mode with variance (α − 1)/β². Move α and β to see when the Gaussian is a good stand-in.

Examples:

━ true gamma density   ╍ Laplace Gaussian   shaded red: the part of the Gaussian that falls below zero, where the gamma has no mass   dotted line: the mode

With the axes fixed, raising β squeezes the density toward zero and makes it taller, and lowering β spreads it out and flattens it. Raising α moves the hump to the right and makes it wider and more symmetric. In both cases the curve can run off the right or top edge: widen the maxima or press the fit button to bring it back into view.

The axes do not change when you move α or β. Set the two maxima yourself, or press the button to fit them to the current curve once.

Mode ŷ = (α − 1)/β
Laplace variance σ² = (α − 1)/β²
True mean α/β
True variance α/β²
Gaussian mass below 0
How the Gaussian is built: the log density is (α − 1) log y − βy plus constants. Its slope is (α − 1)/y − β, which is zero at the mode ŷ = (α − 1)/β. Its second derivative is −(α − 1)/y², which at the mode equals −β²/(α − 1). The Gaussian's variance is minus the inverse of that, (α − 1)/β². For large α the gamma is close to symmetric and the two curves nearly coincide. For α near 1 the gamma is skewed with a long right tail and a sharp left edge at zero, which a symmetric Gaussian cannot reproduce, and the approximation also spills over to negative values. The fit is always best at the mode and worst in the tails.