framework

5 min read

MLOps and Model Serving Readiness Checklist.

Most production model failures are not surprises. They are known gaps that nobody was asked to check before launch: a feature computed differently in serving than in training, a rollback that needs a rebuild, a drift monitor that was never switched on.

This checklist is a go-live gate for classical and LLM endpoints alike, with 45 checks across nine areas. Eighteen are blockers that must be true before launch, and every check names the evidence that proves it, so the review is about links rather than opinions.

It is a gate rather than a guarantee, and it only works if the evidence column is filled.

Download framework Talk to Our Engineers

In depth

What Breaks After Launch Was Usually Visible Before It.

A model without a KPI cannot be judged a success.

If nobody recorded the baseline before launch, there is no way to show the model moved anything, and no way to justify its serving cost later. The checklist makes a business KPI with a baseline, a named business owner and a named technical owner into launch blockers. It also asks for a review date 30 to 90 days after go-live. Unowned models run forever.

In shortUnowned models run forever

Training-serving skew is the quietest failure in production ML.

When features are computed by one piece of code offline and another online, a model can pass every evaluation and still receive inputs it has never seen. The fix is either shared feature code or a skew test, and the checklist treats both as blockers alongside a versioned training snapshot. Most production failures start with data, not with the model.

In shortnot with the model

Latency has to be proven at twice the expected peak.

A load test at expected traffic only proves the model works on a quiet day. The checklist asks for p95 and p99 latency, and for LLMs time to first token and time per output token, at twice expected peak, plus measured cold-start time and throughput per GPU for capacity planning. Capacity discovered under load is discovered by customers.

In shortCapacity discovered under load is discovered by cust…

The safest release is one you can undo in minutes.

A rollback that needs a rebuild is one people hesitate to run, so bad models stay live longer than they should. The checklist requires a tested rollback under five minutes and a shadow or canary release with automated checks before full traffic, with the previous version kept warm until the new one has earned its place. Speed of undo is what makes speed of release safe.

In shortSpeed of undo is what makes speed of release safe

The detail

Three Rules That Turn a Checklist Into a Decision.

A checklist that ends in discussion is a meeting, not a gate. Three rules make the outcome mechanical, and each one leaves a record in the model's registry entry.

Zone · 01

Go

Every blocker is ticked and at least 80% of all 45 checks are ticked. The model proceeds to its canary release as planned, with the evidence links stored against the version. The artefact is a completed checklist attached to the registry entry.

Zone · 02

Go with conditions

Every blocker is ticked but fewer than 80% of all checks are. The model launches, and each open item gets an owner and a date within 30 days. The artefact is the list of open items, each with a name beside it and the date it will be closed.

Zone · 03

No-go

Any blocker is open. The model does not launch until it is fixed and re-reviewed, and any exception needs written sign-off from the accountable executive. The artefact is either the fix or the signed exception, never a verbal agreement in a meeting.

By the numbers

The figures that make it a board-level conversation.

18 of 45

checks are launch blockers, each proven with a named piece of evidence

91%

of model and dataset pairs tested showed quality degradation over time

Under 5 min

rollback time the checklist requires before any model goes live

Inside the report

What you'll take away.

Step 1 - Run the 45-minute review on your next launch

Bring the model owner, a platform engineer and the on-call lead. Fill the evidence column with links rather than statements: no link, no tick.

Step 2 - Treat the 18 blockers as non-negotiable

They cover ownership, data, evaluation, load testing, rollback, release, monitoring, scaling, security, runbook and on-call. None ships open.

Step 3 - Record the decision in the registry

Store go, go with conditions or no-go against the model version, with the evidence and every open item and its date.

Step 4 - Wire the blockers into the release pipeline

Make promotion to production impossible until blocker evidence is attached. A gate that people can skip will be skipped.

Questions

Frequently asked.

Does this apply to LLM applications as well as classical models?

Yes. Most checks apply to both, and the LLM-specific ones sit within them: time to first token and time per output token in load tests, evaluation sets covering accuracy, safety and format, and prompt-injection and output-handling controls.

Why are only 18 of the 45 checks blockers?

Because a gate that blocks everything gets bypassed. The blockers are the checks whose absence turns a bad day into an incident: no rollback, no monitoring, no owner, unproven latency. The rest can ship with a dated plan under the go-with-conditions rule.

Who should be in the review?

The model owner, a platform engineer and the on-call lead, for about 45 minutes. Add privacy or security reviewers when the model touches personal data or is customer-facing. The point is that the people who will be paged at 2am have seen the evidence.

What counts as evidence?

A link to something that exists: a dataset version, a load-test report, a dashboard, a rollback drill log, a signed SLO. A statement that something has been done is not evidence. If the link cannot be produced in the meeting, the check stays unticked.

How do we stop the review becoming a bottleneck?

Automate the checks that repeat. Image digests, load-test results, monitor configuration and registry entries can be attached by the pipeline, which leaves the human review for judgement calls such as slice performance and business impact.

Is drift monitoring really a launch blocker?

Yes. In one study, 91% of model and dataset pairs showed quality degradation over time. Without input drift and output monitoring, the first sign of decline is usually a customer or a business owner, weeks after it started.

What happens after launch?

The model moves to routine operations: a weekly drift and quality review, a monthly cost-per-prediction review, and a success review against its KPI baseline 30 to 90 days after go-live. Rerun the checklist on any major version change.