Service as software

Open the CI config for any service you own. Count the lines.

Somebody wrote every one of them.

Somebody decided which tests run on which paths, which jobs fan out, which caches key on which lockfile, and what happens when a step times out. Most of those decisions were made once, by a person who has since moved teams, and they've been copied forward ever since because nobody wants to be the one who breaks the build.

That file is the product you bought. Everything above it (the part that actually knows whether your change is safe) is still your job.

The claim

You stop configuring and operating delivery software and start consuming it as a service.

Today CI/CD is plumbing. You write the YAML, configure the triggers, and the pipeline does what you told it.

Nothing more. No opinion about your change, because you never gave it one. It runs the list.

The version worth building knows your codebase structure, your dependency graph, your test history and your failure patterns. It analyses the diff, maps the affected services, generates the right validation and runs it, without anyone editing a config file.

You push code. The service validates, tests and deploys. And you pay for successful deployments rather than pipeline minutes.

Stripe doesn't sell you payment processing software. Nobody at your company maintains a config file describing how card networks work; you send a request, and you pay when money moves.

Delivery should work the same way. Mostly, not yet.

Why the minute is the wrong unit

Pricing tells you what a vendor thinks it's selling. A vendor that charges per pipeline minute is selling compute. Its revenue goes up when your builds get slower, when your test suite bloats, when a flaky job retries three times before passing. None of that is malicious. Nothing in the contract rewards the vendor for your pipeline getting better, though, and quite a lot rewards it for your pipeline getting bigger.

I already think the number to watch internally is cost per merged change, not the speedometer. CI compute is a unit cost like any other cloud spend, and the useful question about faster machines is whether they lower the cost per thing that actually ships, or just make the build feel fast while the review queue holds. Fast machines on top of an unbudgeted test suite are a tax on a tax.

Pay-per-successful-deployment is that same idea moved to the other side of the invoice. The vendor gets paid when something you wrote reaches production safely. A failed build costs them money to run and earns them nothing.

A flake, their problem. A test that guards nothing and still takes minutes to run is their problem too, and suddenly they're motivated to find it.

That's the whole point of the pricing change. Incentive, moved to the side of the table that holds the data.

What changed to make this possible

For most of the history of CI, "run everything on every change" was a defensible default. A change or two a day, per human. The diff was the expensive part of the work, and catching a bad one was the job.

Not any more. The diff is now the cheapest part of the work. An agent hands you eight hundred lines where a human wrote eighty. A flake firing weekly at human pace fires several times a day at agent pace. The volume broke the assumption that a fixed list of jobs, maintained by hand, could keep up with what's arriving.

So CI's job moves. Where the spec is precise enough to verify against, CI runs evals that fail loudly when the implementation drifts from what was agreed, and a person reviews the spec rather than the diff. CI also has to defend the review queue, since that's where the constraint went: a machine first pass that checks house conventions and flags missing migrations and unhandled errors before anything reaches a human.

Make diff size policy rather than hope. A large diff approved in ten minutes is an unreviewed change with a green checkmark.

Every one of those jobs needs the pipeline to know something about your code. Which services a change touches. Which tests have historically caught problems on that path and which never have. What "house conventions" means in this repo. A YAML file can't hold that.

A service that watches every build you've ever run can.

The vendor question

Here's what I'd put to every vendor in your delivery stack (CI, deploy, testing, all of it). Tell me when this becomes a service I consume rather than software I operate.

The answer tells you who's building for the next few years and who's still selling 2019.

Listen for specifics.

A vendor who's building the service version will talk about what their system learns from your history, what it decides on your behalf, and what they're willing to be paid on. A vendor who isn't will talk about new integrations, a nicer config syntax, and faster runners. All fine. They're also all ways of making the plumbing more pleasant to operate, which isn't the same thing as taking it off your hands.

Then ask the pricing question directly, whether they'd charge you per successful deployment, and watch the hesitation. A vendor confident in what its product decides should be comfortable being paid on whether those decisions work.

What you still own

None of this means handing over judgment.

You still own what "safe" means for your system. The one-way doors, anything touching money, auth, or data in a way that can't be reversed, still need a person and a slower rollout, and no service should be deciding that classification for you. You still own the spec. The flag too, and the call about when a change is ready to expose.

What you stop owning is the mechanism. Which jobs run, in what order, on which machines, with which caches. That work was never the valuable part, and the people doing it were often your best engineers, pulled off product work because they were the only ones who understood the build.

Until the service exists, treat your pipeline as a product anyway. Give it a roadmap and staff it with a product-minded engineer. Developer experience as a product covers that, and build, buy, or borrow is the test for when to stop building it yourself.

What to measure while you wait

Measure deployments to main, not pull request volume. Measure cost per merged change, not minutes consumed. Measure time from commit to a confident deployment decision, not how often the pipeline runs.

Those are the numbers a vendor selling the service version would be paid on. If you're already tracking them, you'll know within a month whether a vendor's pitch holds up, because you'll be able to hold it against your own baseline.

Not tracking them yet. That's where to start. You can't buy a service on outcomes if you don't know what your outcomes are today.