Measuring an engineering org without lying to yourself

Ask a VP of Engineering how their org is doing and you'll get a number. Deployment frequency, maybe. Lead time for changes. Something from a dashboard someone built two reorgs ago that nobody has questioned since. The number sounds authoritative. It is almost never the whole answer, and sometimes it's actively lying to you.

This isn't a lesson about which metrics are bad. It's about how metrics go bad, on a schedule you can actually predict, and what to do so your dashboard survives contact with the humans it's measuring.

Start with DORA, because it's the framework everyone reaches for first. Deployment frequency, lead time for changes, change failure rate, time to restore. All four measure one thing: how efficiently work moves from an engineer's fingers to a customer's screen. That's genuinely useful information. A team that takes three weeks to ship a one-line fix has a process problem worth finding. A team whose change failure rate is climbing has a quality problem worth finding before it becomes an outage. Elsewhere in this track we call DORA a smoke alarm, not a scoreboard, and that's the frame to hold onto here: it tells you something is burning, not whether the building was worth constructing.

Here's the failure mode DORA can't see. An org can be elite on all four keys, deploying dozens of times a day, changes sailing through with near-zero failures, and be quietly building a graveyard of features nobody asked for. The pipe is fast and clean. What's flowing through it might be worthless. DORA has no opinion on that, because it was never instrumented to have one. It watches the pipe, not the cargo.

SPACE tries to widen the lens: satisfaction, performance, activity, communication and collaboration, efficiency and flow. Most of that is DORA with better manners. The dimension worth taking seriously is the one DORA skips entirely: developer experience. Is the local build fast or a fifteen-minute coffee break? Does a new hire ship something real in week one or week six? Do engineers describe their day as flow or as friction? None of that shows up in a deployment log, and all of it predicts your DORA numbers six months out, because burned-out, blocked engineers eventually stop shipping fast no matter how good the pipeline looks today.

Every one of these breaks the same way

Goodhart's law, stated plainly: once a measure becomes a target, it stops measuring the thing you cared about. This isn't cynicism about your engineers. It's math. If you tell someone their bonus depends on story points, they will produce more story points, and the honest way to do that is to make each story smaller, which they can do without shipping anything more useful. Nobody cheated. The metric just stopped meaning what it used to mean the moment it became a target instead of an observation.

Run the same test on anything you're tempted to put a target on. PR count as a target produces more PRs, chopped finer, reviewed faster and worse. Lines of code as a target (still alive in some org, somewhere, despite everyone knowing better) produces more lines, which is the opposite of what you want from a good engineer. Even deployment frequency, DORA's own flagship metric, degrades the moment someone's performance review depends on it: deploys get smaller and more frequent not because the team got better at shipping, but because that's the cheapest way to move the number. In every case the metric didn't fail. It did exactly what a target does. It got optimized, and the thing it used to stand in for got left behind.

There's a metric almost nobody tracks that would tell you more than any of the above: of everything your org shipped in the last year, what fraction was ever checked against a prediction you wrote down before you built it? Not a retro vibe check. An actual "we expect this to move X by Y, here's how we'll know." Most orgs, including ones with gorgeous DORA scores, will find that number is embarrassingly low. That's the real tell. Speed without a habit of checking your own predictions isn't learning. It's just motion.

What actually belongs on the dashboard

A board doesn't need forty metrics. It needs a handful you'd defend under questioning, plus the discipline to use them correctly. Keep the four DORA keys as your smoke detector: trend lines, not single points, watched for the outlier that says "go ask why," not for a threshold to hit. Add one developer-experience signal, even something as unglamorous as a quarterly survey on whether people feel blocked, because it's the leading indicator the delivery metrics are chasing six months late. Add your confirmation rate: the percentage of shipped work that got measured against what you said it would do. And leave real headroom for anything without a target attached, because the instant a number gets a target, you've started this whole cycle over again on a new metric.

The discipline underneath all of it is simple to state and hard to hold onto under pressure. A doctor doesn't treat blood pressure as a score to beat. They treat it as a vital sign: something that tells you where to look closer. Your dashboard survives exactly as long as you use it that way, to ask "what's the outlier here, what conversation does this start," and not one day longer than the first time someone's comp gets tied to the number instead of the judgment.