The α-geometry, by hand

Companion to the notes on Chapter 4 of Amari's Information Geometry and Its Applications. Back to the notes. Eight small experiments on the chapter's objects: the α-geodesic and the curvature of the simplex, Kurose's Pythagorean identity, the α-mean, apportionment of seats, α-integration, the Tsallis escort metric, the $(u,v)$-divergence and positive-definite matrices. Every number is computed in the browser with the same formulas as code/alpha_geometry.py, and the default state of each widget reproduces numbers printed by that script. Throughout, $D_\alpha[p:q]=\frac4{1-\alpha^2}\sum\{\frac{1-\alpha}2p_i+\frac{1+\alpha}2q_i-p_i^{(1-\alpha)/2}q_i^{(1+\alpha)/2}\}$ as in (3.96): $\alpha=-1$ is $\mathrm{KL}[p:q]$ and $\alpha=1$ its dual.

1. The α-geodesic of the simplex, and what its parameter is

Pick $p$ and $q$ in the triangle of distributions on three outcomes (drag the two dots, or use the sliders) and an $\alpha$. The curve is the normalised α-geodesic (4.27): take $p^{(1-\alpha)/2}$ on the straight line between $p$ and $q$ and rescale to total mass 1. At $\alpha=-1$ it is the straight mixture line; at $\alpha=1$ it is the exponential family through $p$ and $q$. The right panel shows two things the book does not spell out. The un-normalised curve (4.22) of $\mathbb R^3_+$ leaves the simplex: its total mass dips below $1$ in the middle (blue). And the normalised curve is the geodesic of the induced connection only as a set: its affine parameter $\tau$ (orange) is $\int c^2$ with $c$ the normalising constant, not the $t$ of (4.27). Move $\alpha$ to $\pm1$ and the orange curve straightens onto the diagonal.

α-geodesic / total massα = −1 reference / diagonalaffine parameter τ(s)

2. Kurose's Pythagorean theorem

On the simplex the plain Pythagorean identity of Chapter 1 fails for $\alpha\neq\pm1$, and Theorem 4.5 repairs it with a product term: $D_\alpha[p:r]=D_\alpha[p:q]+D_\alpha[q:r]-\frac{1-\alpha^2}4D_\alpha[p:q]D_\alpha[q:r]$ when the α-geodesic $p\to q$ is orthogonal to the $-\alpha$-geodesic $q\to r$. Here $p,q$ are fixed ($p=(0.6,0.3,0.1)$, $q=(0.1,0.3,0.6)$) and $r$ is the point at distance parameter $s$ along the $-\alpha$-geodesic leaving $q$ in the Fisher-orthogonal direction (its tangent scaled to $0.9\min q$ at $s=1$). Slide $\alpha$ and $s$ and compare the plain sum with the corrected one: the correction is positive for $\lvert\alpha\rvert<1$ (positive curvature, like the sphere) and negative beyond, and the formula matches to rounding in every state.

3. The α-mean and why only powers are scale-free

The α-mean (4.33) averages $x^{(1-\alpha)/2}$ and takes the root back. It runs from the maximum ($\alpha\to-\infty$) through the arithmetic mean ($-1$), $\frac14(\sqrt x+\sqrt y)^2$ ($0$), the geometric mean ($1$) and the harmonic mean ($3$) to the minimum, decreasing in $\alpha$ (4.47). Theorem 4.7 says these are the only scale-free means of the quasi-arithmetic form: $m(cx,cy)=c\,m(x,y)$. Test the scale-freeness with the slider $c$: the α-mean passes at every $c$; the quasi-arithmetic mean built from $h(x)=x+x^2$ fails.

4. Dividing seats: the divisor methods are Dα minimisers

Share $n$ seats among states of given populations so that $q_i=n_i/n$ is closest to the population shares $p_i$ in $D_\alpha[p:q]$. The cost is separable and convex in each $n_i$, so the next seat always goes to the state maximising $p_i/d_\alpha(n_i)$ with the divisor $d_\alpha(k)=\lvert(k+1)^b-k^b\rvert^{-1/a}$, $a=\frac{1-\alpha}2$, $b=\frac{1+\alpha}2$. At $\alpha=3$ ($\chi^2$) this is $d=2k+1$, Webster's rule; at $\alpha=-3$ it is $\sqrt{k(k+1)}$, Hill–Huntington; at $\alpha=-1$ ($\mathrm{KL}[p:q]$) the logarithmic mean (Theil–Schrage); $\alpha\to+\infty$ gives Jefferson ($d=k+1$) and $\alpha\to-\infty$ Adams ($d=k$). The page checks the greedy answer against brute-force enumeration of every allocation, and prints the classical rules on the same populations.

5. α-integration: the best way to merge distributions

Given distributions $p_1,p_2,p_3$ with weights $w_i$, the α-integration (4.67) averages $p_i^{(1-\alpha)/2}$ with the weights and renormalises. Theorem 4.9: it minimises the α-risk $R_\alpha[q]=\sum w_iD_\alpha[p_i:q]$. By (4.47) a smaller $\alpha$ is more optimistic (towards the maximum), a larger one more pessimistic. The table lists $R_\alpha[q_{\alpha'}]-R_\alpha[q_\alpha]$: every off-diagonal entry is positive, whatever the weights. Note the book's convention: $\alpha=-1$ is the ordinary mixture and $\alpha=+1$ the normalised geometric mean (the product of experts), the reverse of the sentence after Theorem 4.10.

6. The escort metric is a conformal copy of the Fisher metric

The q-free energy $\psi_q(\theta)=-\log_q p_0$ is convex on the simplex; its gradient is the escort distribution $\eta_i=p_i^q/h_q(p)$, $h_q=\sum p_i^q$, and its Hessian is the $q$-metric (4.92). The book (4.116)–(4.117) says it is the Fisher metric times $\sigma=1/h_q$. Differentiate $\psi_q$ numerically (it is defined only implicitly, by the normalisation) and divide the Hessian entrywise by the Fisher metric in the same $\theta$-chart: every entry is the same number, and it is $q/h_q$, not $1/h_q$. For $q<1$ the escort distribution is pulled towards the uniform one. The divergence of this flat structure is $\tilde D_q[p:r]=\frac1{(1-q)h_q[r]}(1-\sum p_i^{1-q}r_i^q)$ (4.95) with $r$ fixed below; it equals $q\,D_{2q-1}[p:r]/h_q[r]$.

pescort distribution p̃ = pq/hq

7. The $(u,v)$-divergence, and two misprints

A pair of increasing functions $u,v$ gives the Bregman divergence $D_{u,v}[m:m']=\int_0^{m}vu'+\int_0^{m'}uv'-u(m)v(m')$ (4.127) with $\theta=u(m)$ and $\eta=v(m)$ affine and dual affine. Pick a family. For the $(\alpha,\beta)$-divergence the integral is compared with the closed form (4.133); for the $\alpha$-divergence ($\alpha$ of $D_\alpha$ slider) $u,v$ carry the constants of (4.137); for the $\beta$-divergence the page shows (4.140) as printed, which is not even zero at $m'=m$, next to the standard formula that (4.127) produces from $u=m$, $v=m^\beta/\beta$ (the printed $v=m^{1+\beta}/\beta$ of (4.139) gives yet another number). The last line is the Hessian of $D$ in $m'$ at $m'=m$ for the $(\alpha,\beta)$ family: it is $u'v'$, not the $v'/u'$ printed in (4.130).

8. Positive-definite matrices: which divergences survive a change of basis?

For $2\times2$ matrices $P,Q$ the page evaluates six divergences before and after the congruence $P\mapsto L^{\mathsf T}PL$, $Q\mapsto L^{\mathsf T}QL$. Theorem 4.16: the KL-divergence (4.151), $\operatorname{tr}(PQ^{-1})-\log\det(PQ^{-1})-n$, is invariant under every invertible $L$ (it is twice the Gaussian KL of $N(0,P)$ from $N(0,Q)$); so is any function of the eigenvalues of $PQ^{-1}$, such as the $(\alpha,\beta)$-log-det divergence (4.182). The Euclidean, von Neumann and α-divergences of (4.163), (4.166), (4.179) and (4.176) are invariant under rotations only. Turn $L$ into a rotation and they come back.