Maximum likelihood favours complex models

Chapter 2 · Fit a polynomial of order k to the winning times by maximum likelihood. Raise the order and watch the training log-likelihood go up while the prediction for a future Games gets worse.

━ polynomial of the chosen order   ╍ order-1 line   ● data   large dots: predictions for 2016

Training log-likelihood at each order. It never goes down as the order rises, because a higher-order polynomial contains every lower-order one as a special case.