The binomial likelihood

Chapter 3 · The binomial formula P(y | r, N) = (N choose y) r^y (1 − r)^N−y gives the chance of y heads in N tosses when each toss lands heads with probability r. Read as a function of y with r fixed, it is a probability distribution. Read as a function of r with y and N fixed (the observed data), it is the likelihood of r. Same formula, two readings.

Examples:

Number of tosses N

Number of heads y

Head probability r (used in the left plot)

Left: a distribution over y (r fixed)

The bars show P(y | r, N) for every possible y. They add up to 1. The highlighted bar is the observed y.

Right: the likelihood of r (y and N fixed)

The curve is L(r), scaled so that its peak is 1. Its area is not 1. The dashed line marks r from the left plot, and the dotted line marks the peak y/N.

P(y | r, N), the height of the highlighted bar

Sum of the bars

Peak of the likelihood, at r = y/N

Peak value of L(r)

Area under L(r), found numerically

Typical width, √(r̂(1 − r̂)/N)

As a function of r, the likelihood is proportional to r^y(1 − r)^N−y, a hump on [0, 1] that is highest at r = y/N and drops to zero at the ends (unless y = 0 or y = N, when it is highest at an end). The hump narrows as N grows, because more tosses say more about r. The curve is lopsided unless y/N = 1/2. Its area is exactly 1/(N + 1), not 1, so it is not a distribution over r. Multiplying it by a beta prior and normalising gives a beta posterior (Chapter 3).