Theory-inclined papers, with annotations

Frobenius and spectral matrix norms, the Eckart–Young–Mirsky theorem (truncated SVD is the provably-best rank-$k$ approximation in both norms), the 2-norm proof, and an image-compression example

what a matrix can reach (column space) and cannot see (null space), their perpendicular partners, least squares as a right angle with residuals in $N(A^\top)$, the hat matrix and $n-p$ residual degrees of freedom, centring as a residual, the pseudo-inverse and why gradient descent from zero finds the minimum-norm solution, the SVD bases, PCA residual subspaces for OOD, and where each one does work in H-score, LogME, LDA/SFDA, the NTK, spectral contrastive learning and the MICCAI notes

a vector space plus an inner product plus no holes; functions as vectors, with sin x and sin 2x perpendicular and Fourier series; unit balls of different lengths and why only the circle has angles; projection, the perpendicular foot and Pythagoras, and the closest point under three lengths; completeness shown by a sequence closing in on a hole and by the square wave; and where Hilbert spaces appear (least squares, kernels, RKHS, Hilbert–Schmidt, the Fisher metric)

why uncorrelated is not independent (parabola, circle, funnel); squared covariance and $\lVert C\rVert_F^2$ as agreement of two centred Gram matrices; more features catch more dependence, and the Gaussian kernel as every polynomial degree at once; the estimator $\frac{1}{n^2}\operatorname{tr}(KHLH)$ symbol by symbol with a five-point example that honestly fails; the population form and HSIC as MMD between the joint and the product of marginals; bandwidth limits (too wide is linear, too narrow is blind, a checkerboard the median heuristic misses); the permutation test and the $1/n$ bias; CKA and its eigenvalue weighting; COCO, HGR and the H-score as relatives

between/within-class scatter matrices, Fisher's criterion as a generalised eigenproblem $S_w^{-1}S_b$, the $g-1$ rank cap, and the PCA+LDA fix for small-sample singular $S_w$

the SVD characterisation, the projector and distance identities, why `arccos` destroys small angles and the sine fix, and the equivalence with canonical correlations; an interactive 3-D explorer of lines and planes with their principal vectors, the basis-independence of the SVD recipe, and the accuracy trap

$G=XX^\top$ as lengths, angles and distances; rotation invariance and rebuilding points from $G$ (MDS); why it is always PSD and its rank; the dual view $XX^\top$ vs. $X^\top X$ and centring $HGH$; kernel Gram matrices and RBF bandwidth; the NTK as a Gram matrix of gradients; the InfoNCE logit matrix; where each appears in the paper notes; and why forming it squares the condition number

$e^{A}=I+A+A^2/2!+\cdots$ as the linear map that carries every point along the flow $\dot x=Ax$; its properties read off the eigenvalues, why $e^{A+B}\ne e^{A}e^{B}$ unless $A$ and $B$ commute, and scaling and squaring

the chicken-and-egg of latent labels, responsibilities by Bayes' rule (a softmax, and a sigmoid for two blobs), weighted M-step refits with a worked six-point example, why the likelihood never drops (Jensen, a touching lower bound, the $\ln p = \mathcal L + \mathrm{KL}$ gap), k-means as the hard limit, and the step to the ELBO of VAEs

Cheeger and higher-order Cheeger, Markov/Chebyshev/Hoeffding, McDiarmid, Rademacher, KL chain rule, Donsker–Varadhan, HGR, Weyl, Davis–Kahan, Wigner, spherical CLT

a distance as the best gap any judge in a panel can produce between two average scores; judges scored in $[0,1]$ give total variation, 1-Lipschitz judges give $W_1$, judges with height plus slope at most 1 give the Dudley (bounded-Lipschitz) metric, a kernel's unit ball gives MMD; closed forms for two point masses (TV = 1, $W_1 = D$, Dudley $= 2D/(D+2)$, saturating at 2) and for shifted Gaussians; the best ramp judge and why it is optimal for a single hump, cross-checked by an assignment solver; Dudley is not $W_1$ for close distributions unless they are narrow; why bounded panels cannot be "arbitrarily far"; IPMs against $\varphi$-divergences; what a supremum over a panel does not tell you about a downstream task

why a kick in time dt has size √dt (Brownian motion spreads like √t), Euler–Maruyama as gradient descent plus noise, the Ornstein–Uhlenbeck bowl and equipartition (kD/2 of excess loss), the Gibbs law e^(−U/D) from a zero probability current, depth against width (the free energy U + (D/2) log det H), the Fokker–Planck equation and its two time scales, Kramers' law (the barrier sets the exponent, not the destination), detailed balance and two ways to break it (a rotating drift; Hessian-shaped noise gives a round cloud), the path weight (downhill free, uphill e^(−ΔU/D)), ULA's 1/(1 − η/2) bias against MALA, SGLD's extra temperature, SGD's D ≈ ησ²/2B, and annealed Langevin fixing mode weights in score-based sampling

Monge maps vs. Kantorovich couplings, pushforward in plain words, $W_p$ and why it beats KL/JS without overlap (WGAN), Sinkhorn, duality, 1-D and Gaussian (FID) closed forms, minibatch OT for flow matching, and a symbol-by-symbol reading of the discrete OT definition used in data selection

a short hidden Gaussian vector carried along a few directions plus equal noise; the data as a Gaussian with covariance $WW^\top+\sigma^2I$ and the rotation freedom of $W$; the maximum-likelihood solution (PCA subspace, directions shortened to $\sqrt{\lambda_j-\sigma^2}$, noise equal to the average of the discarded eigenvalues); the Gaussian posterior over a point's hidden coordinate as a shrunken PCA projection that tends to PCA as the noise goes to zero; EM without forming the covariance, missing data, factor analysis and the linear VAE; three explorers (build and fit a cloud, drag a point and see its posterior, cut a spectrum to see the noise floor)

a two-coin bag as the running example (marginal $0.25+0.45=0.70$, posterior $0.36/0.64$); why the log of a sum or integral is intractable for a large continuous latent; Jensen's inequality (log of the average vs. average of the log); the bound $\log p(x)\ge\mathrm{ELBO}(q)$; the exact gap $\log p(x)=\mathrm{ELBO}+\mathrm{KL}(q\|p(z|x))$ and why maximising the ELBO pushes $q$ to the posterior; reconstruction minus KL-to-prior as the VAE form; a Gaussian latent where blind sampling fails with dimension and guided sampling does not; an explorer where you drag $q$ and watch the gap

the bell curve whose width is itself uncertain: three routes to the same density, degrees of freedom, heavy tails as a scale mixture of Gaussians, and why the standardised mean with an estimated $\sigma$ follows it

a functional as a height over a space of functions; nudge-and-look-at-the-slope giving the Euler–Lagrange equation and the functional derivative; the shortest path as a worked case; a multiplier per constraint; the Gaussian as the maximum-entropy density for a given mean and variance (with an explorer mixing it toward other densities), great circles as the shortest paths on a sphere (with a globe where pushing the arc sideways always lengthens it), and the exponential family as the general pattern

barrier height as non-convexity along a line; linear connectivity vs a curved path vs linear-up-to-symmetry; the $H!\cdot2^H$ copies of one network and why a straight line between two copies averages unrelated units; a small experiment (straight line 1.81, aligned line 0.0003, one-bend path 0.025; five pairs; width sweep) with a loss heat map through the bend point; a literature map marked read / searched / memory

the best point on a trail as a tangent contour, $\nabla F=\lambda\nabla G$ and the Lagrangian, a fence by a river where $\lambda$ is the price of one more metre, shadow prices and the bead on a wire, maximum entropy giving Boltzmann and the exponential family, $\theta$ and $\eta$ as two addresses linked by the Fisher information, and the limits (candidates, endpoints, singular points, KKT); story after a YouTube video, credited on the page

the score and its variance on a coin (why a fair coin is least informative), Cramér–Rao and efficiency (the median's 2/π), the Fisher matrix as an inverted uncertainty ellipse and blind directions, the local KL expansion, reparameterisation and the Fisher–Rao distance, the Gaussians as a hyperbolic half-plane (geodesics widen, slide, narrow; contrasted with optimal transport), natural gradient and Fisher scoring, Hessian against Fisher against empirical Fisher, the Fisher and the NTK sharing eigenvalues, and how it is estimated in practice

one convex function $\psi$ yields a divergence, a second (Legendre-dual) coordinate system, a metric and two flat structures; softmax logits and probabilities as the running example; the Pythagorean and projection theorems and the alternating (em) minimisation, each rechecked numerically. The printed identity (1.114) has permuted indices, and Theorem 1.4 and the em paragraph pair each projection with the opposite argument slot of $D_\psi$ to what the calculus gives

how far real training departs from the NTK limit, and when

wide networks train as kernel regression in function space

mining general-domain data by NTK alignment with a tiny domain set

why deleting boundary examples improves contrastive learning

why contrastive representations come out approximately Gaussian

contrastive learning is spectral clustering of the augmentation graph

the decomposition of InfoNCE into alignment plus a potential-theory energy

the applied literature that assumes CLIP features are Gaussian, with close reads of Zhou et al. 2025 (CVPR), Venkataramanan et al. 2025 (UAI) and C. Huang et al. 2024 (IJCAI)

an OLS prediction is a signed weighted sum of training outcomes, ŷⱼ = Σᵢ xⱼᵀ(XᵀX)⁻¹xᵢ yᵢ, i.e. attention with the softmax removed, test rows as queries, training rows as keys, outcomes as values and W_Q W_Kᵀ = (XᵀX)⁻¹; ridge, PCR and OLS differ only in the diagonal of one embedding; (XᵀX)⁻¹ is the unique minimiser only if it must serve every outcome vector, and gradient descent on one y reaches the same predictions with a rank-one Ω; the weights sum to 1 but are signed and grow without bound off the data; Regression Block (polynomial features + ridge in place of attention) edges the FT-Transformer by 0.001–0.029 R², credited to the polynomial expansion; Attention Regression trails the random forest on 7 of 8 tables

two ways to differentiate an expectation over a distribution with parameters: move the randomness into a parameter-free noise variable (reparameterization), or weight by the score function (REINFORCE), and when each applies

differentiating through an accept/reject step by integrating out the coin

scoring a source domain with no target labels at all, via a nearest-centroid margin

transfer in a deep linear network from a tiny initialisation: pretraining leaves one direction, so linear transfer has a floor $\sin^2\theta$ against scratch training's double-descent risk, and $\mathcal T_{lt}$ is a phase diagram (positive for few samples, negative for many, positive again in a thin band at $\gamma = 1$; eq. 11 derived in three lines, the same pair of tasks changes sign three times as $n$ grows); fine-tuning keeps the pretrained guess where the data are silent and wins iff $\theta < 60^\circ$ (eq. 15), but real gradient descent carries a scale factor $c = 1 + O(\sigma^2)$ the printed theorem omits (break-even 52° at $\sigma = 1$); Appendix A's Dudley proof fails at step (29) and "any $\delta$" is false beyond 2, Figure 4(a) is a sweep over $[0,\pi]$ and 4(b)'s $W_1$ is sampling noise (≈ 527 for identical laws)

the gradient on the feature extractor lives in the span of the training inputs (so features move only there) and $vv^\top-BB^\top$ is conserved, so a random head drags the features and fine-tuning stalls out-of-distribution while linear probing falls like $\epsilon^2$; the proof of Theorem 3.3 step by step, the two invariants checked by gradient flow (also for the logistic loss), the head-alignment dependence, the toy of Figure 2 run for real (FT wins once the tilt is large), LP-FT, and the slips found in the appendix

imitating fine-tuning by making the target task deliberately harder

task distance as the Fisher–Rao length of a classifier that tracks a task moved from source to target by optimal transport; Definition 2 averages the square root of the KL over inputs ($B\le A$ by Jensen, a Finsler length, not the Riemannian length of Eqs. 4–5) and (12b) is the speed of the weights through a frozen input, not an endpoint distance (2.63 against 0 for a perfectly tracking model); in an exactly solvable logistic toy the minimiser path is exactly reversible, so Remark 3's asymmetry is dynamical and vanishes as the learning speed $\kappa\to\infty$, the SGD length creeps to its limit (6.7% short at $\kappa=3000$) and Remark 1 (uncoupled is longer) holds for $B$ and the loss variation but fails for the Riemannian $A$ at large shifts; Theorem 6's printed $\lambda$ has an extra factor $K$ (exponent $+144$ where the right one is $-1.8$), its exponent has no $N$ (McDiarmid gives $\exp\{-2NK(\epsilon-2\bar R)^2/M^2\}$), the complexity term the proof supports for Theorems 5/7 is $2\,\mathrm{TV}/K$ rather than the length, and the reported Mantel $r$ cannot be reproduced from the printed matrices (Pearson 0.93 against 0.43 in Figure 2; a 4-task test cannot return $p<1/24$)

scoring features by the marginal likelihood of the labels, not the best fit to them

scoring a source model by the likelihood of one hand-built classifier, not the best one

predicting transfer performance without training, via a projection onto the feature subspace

bounding transfer with the conditional entropy of two label sequences, no model needed; the proof's one-term drop costs $\ln(1+\text{other terms}/\text{kept term})$ per image, so the bound is tight for a one-to-one source and loose by $\ln g$ for a fine source with a coarse target; Figure 1 as drawn ties four panels at $\log 2$ (the text's $4\log 2$ needs sixteen labels); in the paper's own Tables 3 and 4, Figure 3's $r=0.78$ recomputes to $0.63$, the target's label entropy alone ties it on Pearson $r$ ($0.62$) though not on rank ($0.66$ against $0.83$), the four CelebA panels in Figure 2 are exactly the top four of forty, and the counted $H(Y\mid Z)$ is biased low by about $0.028$ nats at $17.9$ images per identity, as large as the smallest reported values

whether transferability rankings survive a change of random seed; mostly they do not

seven transferability scores on medical targets, in and out of distribution

transporting CLIP features to text prototypes on the Lorentz manifold

x_t) over spacetime (x_t, t): an exponential family whose chord (Δη)ᵀ(Δμ) is exactly twice a symmetrised KL, so geodesic energies need only the denoiser and one JVP; spacetime is the natural-parameter space (schedule-free), and for Gaussian data the hyperbolic half-plane, which makes DiffED a log of Euclidean distance that saturates; and in a transition-path toy the geodesic reweights wells rather than transporting mass, so its noise schedule, not its route, does the work

a linear VAE is probabilistic PCA, so collapse can be read off the likelihood: it is a local maximum that a too-large noise level creates, not a product of the KL term

the evidence as a prior-averaged likelihood and the Occam factor as the fraction of the prior that fits; Laplace, importance sampling, thermodynamic integration, AIS (arithmetic against geometric mean), variational Bayes ($\log Z=F+\mathrm{KL}$) and nested sampling ($\int_0^1 L(X)\,dX$) derived step by step; the signal-detection log odds ratio by completing the square; and the printed slips found along the way (eq. 23's factor 2, eq. 38's estimator, eq. 69's sign)

moment filtering without the partition function: score matching fits an exp-polynomial belief in one linear solve, Stein's identity closes the moment hierarchy; but the benchmark truth is noise-free, which rewards ignoring the data, and the update as written leaves the mean at the prior

mean, sum and max are 1-, 1- and √d-Lipschitz for EMD, matching and Hausdorff respectively, and each distance's blind spot (duplication, multiplicity, zero vectors) explains five of the six failures at distance zero; the sixth, max against EMD, holds only for unbounded set sizes (constant in [M, √d·M] under a cap), attention fails EMD only on unbounded inputs, the NN_max bound needs the hidden width, and Proposition 3.6 is an equality

a law of robustness for classifiers: swap the Lipschitz constant for the mean distance to the decision boundary, and memorisers need $p\gtrsim nd$ parameters for margins above the free $\sqrt{c/d}$; the ramp surrogate carries every proof, but Corollary 15 drops a covering term and loses its dependence on $p$, and the experiments never cross $p = nd$ on CIFAR-10

non-vacuous few-shot certificates from a finite hypothesis class

a normalization layer that Gaussianizes each unit with a one-step Yeo–Johnson power transform, plus scaled Gaussian noise

treating IID as a property to learn, not assume: find $\phi$ so that $\phi(X_i)$ share one distribution, which turns several causal-inference problems into one representation-learning objective, tested on biomedical data

task complexity as a two-part code with a factorised model, $C=\min_p L_{\mathcal D}(p)+K(p)$, its structure function and $\beta$-Lagrangian, and an asymmetric distance $K(p_{12})-K(p_1)$; whenever the optima are unique the asymmetry is exactly a difference of complexities, and the paper's Figure 1 obeys this to its rounding ($R^2=0.99992$); over all $Q$ the computable Lagrangian is a free energy (the evidence at $\beta=1$), and Theorem 5.4's $O(1)$ is $\tfrac k2\log(N/\beta)-\tfrac k2$, millions of nats at network scale; Theorem 4.4 needs $\beta\ge1$, Lemma A.1 a finite input space and Example 3.5 an index oracle; and sharp phase transitions need a non-convex structure function, so a convex model cannot show them

information in the weights as the optimum of $\mathbb E_Q L+\beta\,\mathrm{KL}(Q\Vert P)$, whose minimiser is the Gibbs distribution; Shannon and Fisher are special codes. The Gaussian optimum is $\beta(H+\beta/\lambda^2)^{-1}$, twice the stated one; the activation Fisher is $\nabla_x f^\top(\beta JF^{-1}J^\top)^{-1}\nabla_x f$, not $JFJ^\top/\beta$, and Eq 10 needs $\dim z\ge\dim x$; isotropic Kramers gives no temperature-dependent preference among equal-loss minima; and in the rebuilt Figure 3 toy the batch-size drop in "Shannon information" is the entropy of which equivalent minimum the seed reached, while $I(\theta;\mathcal D)$ stays flat

SGD as a Langevin particle and a path integral over training runs: the static × reachability split is exactly detailed balance (the path factor is symmetric), and SGD's stationary law is exactly the minimiser of the complexity Lagrangian with β = D and λ² = D/γ (the paper's β = 2λ²γ doubles the weight decay); the curvature correction is (D/2) log|H|, which decides when a sharp minimum loses or turns into a bump; eq. (16) puts end points where Kramers puts the barrier, and on the most likely downhill path the static factor is cancelled exactly; Figure 2's printed matrices still show the static distance ranking sources within a target (r ≈ 0.6); plus a first-reading Q&A (transition probability, the information-theoretic distance, reachability, eqs. 2, 3, 5 and 9 and the SDE line by line, the Fisher in place of the Hessian, a solvable structure function, feature-only transferability scores, dataset against architecture)

the loss landscape as a union of high-dimensional wedges: a straight line between two optima leaves them while a tunnel through their intersections does not; the toy loss run for real (wall 0.29, tunnel 0.07, tunnel alignment cosines 0.24/0.28/+0.01), the intrinsic dimension as a dimension count (hyperplane d >= D - n), m-connectors, averaging as shared short directions (the SWA prediction), and doubts (the tunnel passes through intermediate wedges, Figure 4 not reproduced, axis-aligned wedges vs permutation symmetry)

max-margin linear probes on an imbalanced Gaussian mixture with n/d → δ: training logits are the test Gaussian truncated at the margin, because the d free directions can lift the n training logits by a mean square of (1−ρ²)/δ and truncation is the cheapest way to spend it (Gordon/CGMT); both classes get the same total lift, so each minority point is lifted (1−π)/π times further and the intercept goes negative; rebalancing moves only the intercept and τ_opt has a closed form, but it is negative in several of the paper's own settings, and the π-trend in calibration is almost entirely the prior log-odds that rebalancing discards

replica derivation of the overlap between the true and the learned principal direction, with entries missing at random: $S\to(1-m)S$ in the complete-data formula $R^2=(\alpha S^2-1)/(S+\alpha S^2)$, which is the random-matrix spiked-covariance overlap; the appendix's saddle-point step needs $1-r^2$ where $1+r^2$ is printed, the theory describes zero-filled PCA while the experiments use EM probabilistic PCA, and $N(1-m)^2$, not the tested $N(1-m)$, shares Eq. 2's threshold, so "not sample size" holds against $N(1-m)$ only

scaling the tumour's non-conformity score by a distance-to-vessel weight so prediction sets grow near vessels; sets can only get larger, so validity is inherited, while local coverage is observed rather than guaranteed

sampling plausible cup and disc masks, turning each into a rim thickness curve, and averaging the classifier's votes; the claimed gain over a deterministic U-Net is clearest only for the seed ensemble

separating annotator error from the true segmentation with a trace penalty on per-pixel confusion matrices (no HSIC); the per-pixel theorem needs column dominance, and without it the trace is smallest at the label swap

one latent variable per resolution level, each conditioned on the coarser one, so a segmenter can produce many plausible masks matching annotator disagreement

a conditional VAE whose latent code, injected late into a U-Net, selects one coherent plausible mask; learns variant frequencies; judged by the generalised energy distance

ResNet blocks as Euler steps of a velocity-field ODE, with spectral normalisation for Lipschitz control; fewer folded voxels than VoxelMorph at slightly worse matching

a warp as the end of a flow of a smooth velocity field, chosen to minimise path energy plus mismatch; the Euler–Lagrange equation step by step, the Sobolev (Hilbert) gradient against the $L^2$ gradient, constant-speed re-timing, and the image metric; the experiments use a milder smoothing operator than the diffeomorphism guarantee asks for, and the shorter paths than Christensen's method cost 10 to 80 times the run time

a lesion is found only if its residual beats the reconstruction error at healthy pixels, so the score is blind to lesions near common tissue intensity; a sharper autoencoder also reconstructs the lesion; the one inequality it rests on is loose and the evidence is simulated discs

passing one model from hospital to hospital instead of averaging weights; the algebra only unrolls the FedAvg update, the gains are fractions of a Dice point, and the ensemble variant is compared with single models

the core mathematical construct of each of 26 MICCAI 2026 DA/DG/TTA/OOD papers (Mahalanobis residuals, deep EM, OT and Schrödinger bridges, flow-matching TTA, InfoMax, variational logit energies, GRPO), with a construct map, one toy-computed figure per paper, and a *What to watch* per paper

the core mathematical construct of each of 11 MICCAI 2026 papers tagged image quality assessment (cross-sectional self-consistency for label noise, prototype-guided flow matching, few-shot prototypical networks with FiLM and gradient reversal, a failure-driven MLLM data engine, loss-GMM pseudo-label filtering, generalised Lehmer pooling, Dice under class imbalance, set encoders for missing metadata, non-negative PU learning, masked autoencoders with LoRA, CORN and concept MIL), with a what-is-graded table, a construct map, one interactive widget per paper, and a *What to watch* per paper