Series: Measuring what matters
- Part 1: proxies and their failure modes
- Part 2: session length is a trap
Nobody wants “clicks”. We want people to find what they came for. Clicks are a proxy, and proxies fail in a handful of recurring ways.
Four failure modes
- Saturation: the proxy stops moving while the goal keeps improving.
- Inversion: more of the proxy means less of the goal (more searches because results got worse).
- Gaming: the proxy becomes a target and detaches from the goal.
- Heterogeneity: the proxy means different things for different groups.

The defence is not a better proxy but several, chosen so that their failure modes do not overlap, and a habit of checking them against qualitative evidence.1
- Interviews and support tickets are slow and noisy, but they rarely fail in the same direction as an engagement metric. ↩