On-call that doesn't burn people

The rule I don't make exceptions to: the people who built it carry it.

Not a dedicated ops rotation.

Not a follow-the-sun team receiving alerts about code they've never read.

The engineers who wrote the thing are the ones whose phone goes off, and that stays true however large the company gets.

The reason isn't fairness, though it is fairer.

It's that the feedback loop between "how I built this" and "what it does to me at 3am" is the single most reliable quality mechanism in software, and every alternative arrangement severs it.

A team that doesn't carry its own pager will, over time, ship things that are painful to operate. Not through malice. Because the pain is invisible to them and the deadline isn't.

Hand the pager to a separate group and you get exactly what you asked for.

A group who can't fix anything, escalating to people who can, at 3am, with a delay in the middle.

Rotation design

Six people minimum for a sustainable rotation.

That gives one week in six, roughly the point where being on call stops dominating your entire month.

Below six it degrades quickly.

One week in four means you're always either on call, just off it, or about to be, and people start planning their lives around that. One in three isn't a rotation. It's a job with intermittent sleep.

If a team of four owns something that pages, either they own too much or the system needs work before the rotation does.

Weekly shifts, handed over at a fixed time, with an actual conversation.

The handover matters more than people expect. What's currently unstable. What fired last night. What you should know before Tuesday.

Five minutes, and it stops the same alert being investigated from scratch by three consecutive people.

Primary and secondary, where the secondary is real backup rather than escalation theatre. The primary is going to miss a page eventually, and the correct response is a second phone rather than an incident review about the first one.

Follow-the-sun sounds like the humane answer and mostly isn't, at least below a few hundred engineers.

It requires teams in three regions who each understand the same systems well enough to act. What you usually get instead is a night-time team who can restart things and file a ticket, which trades one team's sleep for everybody's mean time to resolution.

Genuine multi-region engineering with genuine shared ownership is excellent. A support team in a cheaper timezone is a support team in a cheaper timezone. Call it that.

The alert that should have been a ticket

Most on-call pain isn't incident pain.

It's alert pain, and it's self-inflicted.

Every page should mean one thing. A human needs to act, right now, or something meaningfully bad happens.

Anything else is a ticket, a dashboard, or nothing at all.

Apply that literally and most alert catalogues shrink by half.

Disk at 80% on a node being replaced tomorrow: not a page. CPU spike that resolved itself: not a page. A batch job that failed and will retry: not a page.

An error rate that's been "elevated" for six months: definitely not a page. And the fact it's still there means somebody has already learned to ignore it, which is where ignoring everything starts.

The number I'd hold: fewer than two pages per night shift on average, and a night with zero should be normal rather than notable.

Above that you aren't running an on-call rotation. You're running a sleep deprivation programme with a ticketing system attached.

Two habits keep it there.

Every page gets a quick verdict at handover. Was this actionable, yes or no.

And a no means somebody owns fixing or deleting that alert this week, with the same seriousness as a bug. Alert hygiene decays continuously, so it needs a continuous counter-force.

Paying for it, and time back

Compensation for on-call is genuinely contested, and the answer varies by country and by company stage.

What isn't contested, or shouldn't be: the time has to come back somehow.

The minimum, non-negotiable version.

Up at 3am means you're not expected online at 9am, and the manager says so rather than leaving the person to ask. People will not take it if it needs asking.

And a heavy week means a light next week. Planned, rather than heroic.

Beyond that, pay for it if you can. Particularly out-of-hours coverage.

An on-call allowance makes the cost of a noisy system visible in a budget line, which does more to motivate fixing it than any amount of discussion ever will. One of the few cases where putting a price on something improves the engineering.

What doesn't work is treating on-call as a professional obligation serious engineers simply accept.

That norm is quietly a filter. It selects for people without caring responsibilities, and the people it filters out won't tell you that's why they left.

Feeding the pain into the roadmap, with teeth

Every company says on-call pain informs the roadmap.

Very few have a mechanism. Without one it's an aspiration, and it loses to the quarterly plan every time.

The mechanism I'd use.

On-call load has a threshold, and crossing it converts automatically into roadmap capacity. No permission required.

Something like this. Paged more than a set number of times outside hours in a month, and the next sprint's first item is the top cause, chosen by the people who were woken.

Not proposed. Not prioritised against features. Taken, automatically, because the alternative has already been paid for in sleep.

The automatic part is what gives it teeth.

A rule requiring a manager to make the call each time gets overridden in the quarter when it matters most, which is always the quarter where the system is worst.

Pair it with a standing report going to the same audience as the delivery numbers. Pages per week, by team, with a trend line.

Anything not reported alongside delivery gets treated as subordinate to delivery.

What the rotation tells you

Look at the on-call log for a quarter and you have a map of your architecture's real weak points, ranked by what they cost humans.

Better information than any architecture review, because it's empirical and nobody's opinion is in it.

Three patterns worth reading.

The same service, repeatedly, means a design problem rather than a bug backlog.

Pages clustering around deploys mean your pipeline's safety net has a hole.

And pages only one person can resolve mean you have a knowledge concentration that's about to become an availability problem the week they go on holiday.

That last one is the one people fix last and regret most.