SLOs you'll actually honour

The test for whether you have a service level objective is whether you've ever stopped something people wanted in order to defend it. Writing it down proves nothing.

I have, and that was the moment the whole practice became real for me rather than theoretical.

Before then, an SLO is just a number on a dashboard, discussed when it happens to be green and explained away when it turns red.

Two or three, not fifteen

Almost every SLO programme fails the same way.

Too many objectives. Defined by the people who own the systems instead of by what a user would notice. Tracked in a spreadsheet nobody opens by month four.

Pick two or three per product surface, and no more than that.

Each should be something a customer could describe in one sentence, and the phrasing matters a great deal, because it decides what you'll end up defending.

"The API is up" fails that immediately, because users experience whether their particular thing worked and uptime is entirely invisible to them.

Closer to it. A search returning in under a second for 99% of requests. A checkout completing successfully 99.9% of the time. The nightly export landing before 6am on 99% of days.

The test for a good indicator is simple enough: if this is green and a customer is angry, you have the wrong indicator.

That check kills most availability metrics. A service can be perfectly up and completely useless.

It also kills anything measured from inside your own infrastructure, since the customer's experience includes the network, the browser, and the third party you don't control.

Picking the number honestly

Start from what you're currently delivering.

Measure four weeks, look at the number, and set a target only a small step better than the reality you just measured.

The failure mode is picking an aspirational number because it sounds professional. Four nines, say.

Four nines means about four minutes of failure per month. If you're currently at two nines, adopting it commits you to an engineering programme you haven't planned, and everybody quietly stops looking at the number by the second month.

The other direction matters too, and almost nobody works it.

If you're already at four nines on something the business doesn't need four nines for, you're over-delivering reliability at a real cost, and the correct move is to spend some of that margin on shipping faster.

That's the least intuitive part of the whole framework, and it's also the part that turns it into a business tool instead of an engineering wish list.

Ask it in the business's own language. How much would this failing for an hour actually cost us, in revenue, in support load, in churn.

Some systems genuinely warrant heroic reliability. Most internal tools don't, and treating them as though they do is where an enormous amount of engineering effort goes to die.

The error budget policy is the whole mechanism

An error budget is the arithmetic complement of the objective.

At 99.9% you have roughly 43 minutes a month to spend, and "spend" is exactly the right verb, because it's a resource for taking risks with. Not a target to beat.

Coming in well under budget every month means you were too cautious.

All of which has precisely zero force without a written policy saying what happens when the budget runs out, agreed in advance, by a group that includes the people who own the roadmap.

Mine is short. One sentence, really.

Budget exhausted means the next work that team picks up is reliability work on the cause, until the budget recovers. No discussion, and no proposal to be weighed against a launch.

The decision was already made, back when everybody was calm.

It has to be pre-agreed because the conversation is unwinnable in the moment.

The moment always coincides with something important shipping, everybody's judgment is contaminated by the pressure they happen to be under, and reliability loses every single time it gets re-litigated in those conditions.

Writing it down in advance is the only way to make a promise to your future self that survives contact with a bad week.

The policy needs an off-ramp to stay credible. An explicit way for the CEO or whoever to override it, in writing, with a reason attached.

A rule with no override gets broken quietly, and a quiet break does more damage than a documented one.

It also needs an expiry, so budgets reset monthly and one bad month doesn't hold the team hostage forever.

Negotiating the target with the business

Do this once, properly, with the person who owns the revenue. Easier than engineers expect.

Frame it as a choice. Never a request.

"We can hold this at 99.9%, which costs roughly this much engineering attention, or at 99%, which frees some of that up for features. Below 99% we start seeing support volume and churn."

That's a trade expressed in their own currency, and making trades is what they do all day.

Asking for reliability investment as a general good gets you nowhere. It reads as insurance against an unspecified event, and it competes badly against a specific feature for a specific customer.

So bring evidence from your own history. The last three incidents. What each cost in support tickets, refunds, deals delayed.

Reliability arguments built on real incidents inside your own company land completely differently from arguments built on principle.

What to do before you're ready for any of this

Under about twenty engineers, formal SLOs are usually premature.

Not enough traffic for the percentages to mean anything, and not enough people to run the programme.

What you want instead is the precursor. One measurement, from the customer's side, of whether the main thing works.

A synthetic check performing a real transaction every few minutes, paging somebody if it fails twice in a row.

A day of work, and most of the benefit.

The rest comes later. Usually the day somebody asks what "reliable enough" means and you notice three people in the room have three different answers.

The failure I'd watch for

An SLO green for eight months with nobody looking at it. Evidence of very little.

Usually it means the indicator drifted away from what users care about, or the measurement broke and has been reporting success by default ever since.

So check the measurement itself once a quarter, by breaking something on purpose in a controlled way and confirming that the number actually moves.

A monitoring system nobody has ever seen fail is a monitoring system nobody has tested.