Most organizations now use AI, and very few can operate it well. MLOps is the set of engineering practices that gets machine learning models into production and keeps them accurate, safe and affordable, but the two reference models teams cite, Google's levels 0 to 2 and Microsoft's levels 0 to 4, were written for predictive models.
The CNCF's 2025 Annual Cloud Native Survey found only 7% of organizations deploy AI models daily. This paper extends those models into one five-level scale across ten dimensions that covers predictive ML, LLM applications and agents, with a scorecard, a platform comparison and a two-quarter roadmap.
It draws on published reference models, analyst and survey data and vendor documentation current to November 2026. It is not a benchmark of tools on a shared test, and it reports no client results.
The scorecard rates ten dimensions from 0 to 4, and your lowest score is your effective maturity level. For LLM and agent workloads, three of them carry the most weight because they map to the value, risk and cost causes in Gartner's agent forecast. Each one is fixed by a named, checkable artifact.
LLM output is open-ended and non-deterministic, so teams combine reference answers, rubric-based LLM judges and human annotation. Agents also need task-level tests: did the agent finish the workflow, call the right tool with valid arguments, and stop to ask when it should? A team with a 3 in serving and a 0 in evaluation ships fast but cannot tell whether a release made answers worse. The artifact is an eval set built from real traffic that every release must pass.
About one in five respondents in McKinsey's 2026 survey say operating costs, including tokens, now constrain their AI use. At Level 3, token and GPU budgets raise alerts the way latency does. At Level 4, a gateway routes simple requests to smaller models and cost per business outcome is tracked for each release. The artifact is spend tagged per model or app, with a token budget and a named cost owner.
Agents act in other systems: they write to databases, send email or call payment APIs. NIST's Generative AI Profile maps actions for generative AI risks to the AI RMF functions Govern, Map, Measure and Manage. For agents, the practical controls are scoped credentials, allow-listed tools and approval for irreversible actions. The artifact is an immutable action log for every agent, with controls mapped to the NIST AI RMF.
Rate data, experimentation, CI/CD/CT, registry, serving, monitoring, evaluation, governance, cost and people from 0 to 4 for what is true in production, separately for predictive ML, LLM apps and agents.
Your lowest score is your effective maturity. If your average exceeds it by more than 1.5, investment is unbalanced, so pick one workload class and write exit criteria before buying tools.
Pair a core ML platform such as SageMaker, Vertex AI, Azure Machine Learning, Databricks or MLflow with Kubeflow, then add an LLM tracing and evaluation tool such as Langfuse, LangSmith or Arize.
Run baseline, foundations, evaluation, monitoring and cost, then governance and review, each with exit criteria. Rescore at week 26 and report the result with the next target.
MLOps, short for machine learning operations, is the practice of building, deploying, monitoring and governing ML models as production software. It extends DevOps to data and trained models, adding continuous training, data validation and drift monitoring, as Google's reference model describes.
In 2026 most teams also apply it to LLM prompts, evaluation sets and AI agents, which is the scope this paper's maturity model covers.
It depends on the model you use. Google defines three levels, from manual process to CI/CD pipeline automation. Microsoft defines five, ending with drift-triggered retraining. This paper uses five levels, Ad hoc, Repeatable, Managed, Automated and Continuously governed, scored across ten dimensions that include evaluation, governance and cost for LLM and agent workloads. Level 3 is the right target for most production workloads.
DevOps automates how application code is built, tested and released. MLOps adds data and trained models, plus continuous training, data and model validation, and drift monitoring, because model behavior changes when input data changes.
LLMOps, also called GenAIOps, reuses those foundations and adds prompt versioning, retrieval pipelines, output safety and token cost governance. Agent operations add tool permissions, action logging and task-level evaluation.
Start where your data and cloud contracts sit: SageMaker for AWS, Vertex AI for Google Cloud, Azure Machine Learning for Microsoft, or Databricks if your data lives there. Teams that want portability use MLflow and Kubeflow.
No single product covers all ten dimensions well, so most enterprises add an LLM tracing and evaluation tool such as Langfuse, LangSmith or Arize alongside the core platform.
About two quarters for one workload class, if you limit scope to the two or three lowest-scoring dimensions and set exit criteria first. A Level 1 to 2 move mostly means a registry, tracing and an evaluation gate.
A Level 2 to 3 move mostly means pipeline-driven promotion, evaluation gates, canary releases with automatic rollback and on-call alerting.
MLOps as a service means buying MLOps capability instead of building all of it: a managed cloud platform, a hosted observability and evaluation tool, or an engineering partner that builds and runs the platform with you.
Two questions decide it. Can a platform team own on-call for ML infrastructure, and must data and traces stay in your cloud account? If no and yes, pick a managed cloud platform with self-hosted tracing.
You get the five-level model, a ten-dimension self-assessment scorecard, a platform comparison by dimension, a decision table for where to invest first and a 26-week roadmap. The evidence comes from Google's and Microsoft's reference models, Gartner, McKinsey, Stanford and CNCF data, NIST frameworks and vendor documentation current to November 2026.
It does not benchmark tools on a shared test, so confirm current features with each vendor before buying.
Drop your details and we'll send The MLOps Maturity Model straight to your inbox - no spam, unsubscribe anytime.
Bring one production workload, whether a predictive model, an LLM application or an agent, and the people who run it. We will work through the ten-dimension scorecard with you and agree which two or three dimensions to fund over the next two quarters. SECTION 7 - FAQ - 5 to 8 questions
Book a maturity review