The Metric Was the Design
Ask an engineer what a recommendation feed optimises for, and the answer is usually a number: session length, click-through, time-on-app. Ask what the feed is for, and the answer is almost always something else entirely — “helping people find what matters to them.”
Those two answers are rarely the same sentence, and the gap between them is not an implementation detail. It is the entire architecture.
A ranking algorithm doesn’t discover what matters to a user. It maximises whatever metric someone chose to hand it. If that metric is engagement, the algorithm will find — reliably, tirelessly — whatever keeps a person scrolling, whether or not that thing is what they’d say, out loud, they wanted. None of that was the intent. It was simply the fastest path the optimisation found to the metric it was actually given.
This is intent preservation’s least forgiving test case. Somewhere upstream, a business intent got stated in plain language — “help people find what matters.” Somewhere downstream, that sentence became a loss function to minimise. Between those two points, the actual meaning of “matters” quietly got replaced by “measurable,” because measurable is the only thing a ranking function can act on. Nobody voted for that substitution. It happened by default, the moment nobody architected against it.
The fix isn’t a smarter algorithm. It’s treating the choice of metric as the architectural decision it always was — worth the same defensible-trade-off discipline as any other. What did you choose to optimise, and what did you knowingly leave unmeasured because it was harder to quantify? Whose experience gets sacrificed when the metric and the stated intent diverge, and did anyone decide that on purpose, or did it just happen?
A feed that optimises purely for engagement isn’t a bad feed by accident. It’s exactly the feed its metric was designed to produce. The architecture was never neutral — the metric was the design, all along, whether or not anyone wrote it down as one.
This is the same discipline the manifesto and the books apply everywhere else: judging a decision by whether it can be defended, not by whether it happened to work.