Set aggregators and their distances

Companion to the notes on Nikolentzos & Skianis, On the Lipschitz Continuity of Set Aggregation Functions and Neural Networks for Sets (ICLR 2026). Back to the notes.

A set network pools its element embeddings with a sum, mean or max. How far the pooled vector can move, per unit that the input set moves, is the aggregator's Lipschitz constant. The answer depends on how "the input set moves" is measured. The three widgets below let you watch that dependence directly.

1. Two sets, three distances, nine ratios

Blue is $X$, orange is $Y$. Click empty space to add a point to the selected set, drag a point to move it, and switch to Delete to remove one. Up to 6 points per set. The table shows $\lVert f(X)-f(Y)\rVert/d(X,Y)$ for every aggregator $f$ and distance $d$. It is green where the paper's theorem bounds it (bound underneath), red where the distance is 0 but the aggregates differ (a blind spot), and amber for max against EMD, which is bounded only because the size is capped.

Presets:
XY○ mean  □ sum  △ max (outlined in each set's colour)⊕ origin, where matching's phantom zeros sit

2. Push each counterexample to its limit

The paper proves each "not Lipschitz" entry with a family of pairs whose ratio grows without bound. Pick one, then move the slider. Watch how it grows. Five families approach a pair at distance exactly zero (a blind spot), and blow up as $\varepsilon\to0$ at a fixed scale. Max against EMD grows only with the set size $M$. Attention against EMD grows only with the input scale $c$. All in $d=3$, computed with the real distances, not formulas.

3. Add one point (Proposition 3.6)

A cloud $X$ of $n$ points and one extra point $\mathbf w$ (red, drag it). The Hausdorff distance to $X\cup\{\!\{\mathbf w\}\!\}$ is $\mathbf w$'s distance to its nearest neighbour. The EMD is exactly $\frac{1}{n(n+1)}\sum_i\lVert\mathbf v_i-\mathbf w\rVert$. The paper states this as an upper bound, but it is always an equality. For $n\le 10$ the page also solves the transport problem directly, so you can check. Drag $\mathbf w$ far away and compare how the mean (hollow circles) and the max (triangles) respond.

Everything here is exact. EMD between sizes $m$ and $n$ is solved by repeating each point of $X$ $L/m$ times and each point of $Y$ $L/n$ times ($L=\operatorname{lcm}(m,n)$) and running the Hungarian algorithm; the matching distance pads the smaller set with zero vectors. The same code in Python is in code/set_lipschitz.py.