Added time at the end of a half is not arbitrary, though it is discretionary. It represents an attempt to recover playing time that the clock kept running through.
The clock never stops
Football runs a continuous clock, unlike sports that pause for every stoppage. The referee is the only person keeping track of what has been lost.
Everything that interrupts play while the clock runs is meant to be counted: substitutions, injuries, goal celebrations, disciplinary incidents and deliberate time wasting.
The fourth official displays a minimum figure, and the referee may extend beyond it if further stoppages occur during the added period itself. The board is therefore a floor rather than a countdown, which is why matches sometimes run past the number shown.
Why the figure rose
For years the accounting was loose, and the added time bore little relation to the interruptions that had actually occurred during the half.
Governing bodies instructed officials to count more rigorously, particularly the time consumed by celebrations and by substitutions late in a match.
The immediate effect was a sharp increase in displayed stoppage time, which looked like a rule change but was really a change in enforcement. Nothing in the laws had been rewritten; the existing instruction was simply applied as written.
The behaviour it is meant to correct
A leading team has an obvious incentive to slow the game down, and the tools are mundane: slow restarts, unhurried substitutions, prolonged treatment.
If none of that is recovered, the reward for wasting time is a shorter match, which rewards the tactic and punishes the side trying to play.
Counting it back removes the reward. The delay still costs the trailing team momentum, but it no longer shortens the amount of football played.
The knock-on effects
Longer halves mean more distance covered, and squads have had to account for that in conditioning and in how substitutions are timed.
Late goals have become more common, simply because there is more football in which they can occur, which changes the value of a narrow lead.
Broadcasters and stadium operators have adjusted schedules too, since a match that regularly runs long affects everything arranged around it, from transport departures to the start of the following programme.
Why it remains an estimate
Precision is impossible without stopping the clock, and stopping the clock would change the character of the game in ways the sport has resisted.
Proposals for a stopped clock surface periodically and are usually set aside, partly because continuous play is central to how football feels.
The compromise is a judgement call by an official, applied more strictly than before but still a judgement rather than a measurement.


