first-course-ml

A First Course in Machine Learning

Notes by chapter (work in progress).

Chapter 1: Linear modelling, a least-squares approach

• Interactive: fit a line to the Olympic data
• Vector and matrix notation, and differentiating with it

Chapter 2: Linear modelling from a probabilistic view

• Interactive: likelihood of a Gaussian linear model
• Interactive: maximum likelihood favours complex models
• Interactive: curvature of the likelihood (the Hessian)
• Interactive: the quadratic bowl (log-likelihood surface)
• Interactive: predictive error bars
• Maximum likelihood for the Bernoulli parameter

Chapter 3: The Bayesian approach

• Chapter 3 summary: the Bayesian approach
• Interactive: the beta density
• Interactive: the binomial likelihood
• The product of two Gaussians: derivation
• Interactive: the product of two Gaussians
• Mean and variance of a beta random variable
• Posterior with a flat prior

Chapter 4: Bayesian inference

Overview

• Notes: Chapter 4 summary

The running example: the binary response model

• Interactive: the binary response dataset, with prior draws

Prior, likelihood and the unnormalised posterior

• Interactive: the likelihood read two ways (density of the data, likelihood of the weight)
• Interactive: evaluating the numerator of Bayes’ rule at one w
• Notes: the numerator of the posterior and what is intractable

MAP and Newton–Raphson

• Interactive: Newton–Raphson step by step

Laplace approximation

• Interactive: the gamma density and its Laplace approximation
• Interactive: Laplace approximation for the binary response model

Exercises

• Notes: Exercise 4.1, the Laplace approximation is exact for a linear Gaussian model
• Notes: Exercise 4.2, the Laplace approximation to the coin posterior
• Interactive: Exercise 4.3, the coin posterior and its Laplace approximation

Chapter 5: Classification

• Chapter 5 summary: classification

Chapter 6: Clustering

• Chapter 6 summary: clustering

Chapter 7: Principal components analysis and latent variable models

• Chapter 7 summary: PCA and latent variable models