Mixture of experts is a neural network design where the model is split into many smaller subnetworks, called experts, and a router decides which few experts handle each input instead of running the whole network every time. Only a small slice of the total parameters activates for any given token or example. The rest sit idle until the router calls on them for a different kind of input. This is different from a standard dense network, where every parameter does work on every single input regardless of whether that work is needed.
The reason mixture of experts exists is that dense scaling ran into a wall. Making a language model smarter by simply adding more parameters also multiplies the compute needed to train and run it, and at some point the cost curve stops making sense for the quality gained. Teams needed a way to grow a model's total capacity, meaning how much it can learn and store, without growing the amount of math it does on every single input by the same amount. Mixture of experts breaks that link between total size and per-input cost.
What distinguishes mixture of experts is the gating network, a small learned component that looks at each token and decides which experts should process it, usually the top one or two out of dozens or hundreds available. During training, the model learns both the experts themselves and the routing decisions at the same time, so certain experts naturally start specializing in certain kinds of patterns, whether that is a language, a topic, a type of reasoning, or something less human-interpretable. The result is a model with a huge parameter count on paper but a much smaller "active" parameter count on any given pass, which is the number that actually determines inference speed and cost.
By 2026, mixture of experts has become the default architecture behind most of the largest and fastest production language models, replacing dense architectures at the frontier almost entirely. Vendors adopted it because it lets them keep pushing total model capacity up while keeping serving costs from spiraling out of control, which matters enormously once a model is answering millions of requests a day rather than sitting in a research paper. For any team buying or building on top of large models, understanding this shift explains a lot of the pricing and performance patterns visible in the market right now.
This page covers how mixture of experts routing works, how it compares to dense models, why it matters for anyone scaling AI systems, where it fits and where it does not, and how a technical team can evaluate or adopt it with a clear head. The durable idea underneath all of it is simple: not every input needs the full model, and building a system that recognizes this can be more efficient than one that treats every input the same. Grasping that idea lets a team read model cards, compare vendor claims, and make architecture decisions instead of just taking benchmark scores at face value.
A mixture of experts layer replaces a single large feed-forward block inside a transformer architecture with a collection of smaller feed-forward blocks, the experts, sitting side by side. Each expert has its own weights and, in principle, can learn to specialize in different kinds of input patterns. Attached to this layer is the router, sometimes called a gating network, which is itself a small trained layer that takes the incoming token representation and outputs a score for every expert. The system then picks the top-scoring expert or experts, commonly one or two, and sends the token only to those. The number of experts selected per token, often called top-k routing, is itself a design choice, and different models pick different values depending on how much they want to trade specialization for combined coverage across experts.
The token gets processed by its chosen experts, and their outputs get combined, often with a weighted sum based on the router's confidence scores, before moving to the next layer of the network. Every other expert in that layer does nothing for that token. Multiply this across every token in a sequence and every mixture of experts layer in the network, and you get a system where the total parameter count might be enormous, sometimes many times larger than a comparable dense model, while the actual computation performed per token stays close to what a much smaller dense model would need. Different tokens in the same sentence can end up routed to entirely different experts, which is part of why a single forward pass through a mixture of experts model can touch a surprisingly wide slice of the total network even though any one token only sees a narrow slice of it.
Training this kind of network is trickier than training a dense one. Left alone, a router will often collapse toward favoring a handful of experts because those experts get more training signal early on, get better faster, and then get picked more often, starving the other experts of the practice they need to become useful. To prevent this, mixture of experts training typically adds auxiliary loss terms that penalize uneven routing and push the model toward spreading tokens across experts more evenly. Getting this balance right is one of the harder parts of building these systems well, and it is a big part of why early attempts at the architecture were less reliable than more recent ones. Some teams also add noise to the routing scores during training specifically to discourage this kind of early collapse, giving underused experts a fairer chance to receive tokens and develop useful specializations before the router settles into a stable pattern.
The attention mechanism layers in these models typically stay dense, meaning every token still attends to relevant context the normal way. It is usually just the feed-forward layers that get split into experts. This matters because it means mixture of experts is not a replacement for the core attention machinery that lets a model understand context and relationships between words; it is a modification to a different part of the network, aimed specifically at how much computation happens once attention has done its job of figuring out what matters. A model can be dense in its attention layers and sparse in its feed-forward layers at the same time, which is exactly the arrangement most production mixture of experts systems use today, since attention computation is a smaller share of total cost than the feed-forward blocks it sits alongside.
A dense model runs every parameter on every input, full stop. If a dense model has 70 billion parameters, processing a single token involves computation tied to something close to all 70 billion of them, every time. This is straightforward to reason about and straightforward to train, which is why it was the standard approach for years. It is also expensive to scale, because doubling the parameter count roughly doubles the compute cost of every forward pass, with no shortcuts available. This straightforward relationship between size and cost is also what made dense models easy to benchmark and compare for so long: a bigger dense model almost always cost proportionally more to run, so buyers could reason about the trade-off with simple arithmetic.
A mixture of experts model with, say, 400 billion total parameters but only 40 billion active per token gives a team access to a much larger pool of learned knowledge and pattern recognition while paying compute costs closer to what a 40-billion-parameter dense model would cost. This is the core trade a company makes when choosing mixture of experts: more total capacity and often better quality on complex tasks, in exchange for a more complicated system to train, serve, and debug. The quality gains are real but they are not free in engineering terms.
Memory is the other side of this trade that gets less attention than it deserves. Even though only a fraction of parameters activate per token, all of the experts still need to sit in memory somewhere, because the router might call on any of them at any moment depending on the input. This means a mixture of experts model can require more total memory to host than its active parameter count would suggest, even while its compute cost per token stays low. Teams evaluating vendor infrastructure or planning their own deployments need to account for this, because "compute-efficient" is not the same claim as "memory-cheap." Serving a mixture of experts model efficiently across multiple machines also raises its own engineering questions, since experts often need to be distributed across different pieces of hardware, and tokens routed to an expert on a different machine than the one currently processing them introduce communication overhead that a dense model, with all its weights sitting together, never has to deal with.
Dense models still have a place, particularly for smaller models meant to run on modest hardware or for cases where the added complexity of routing, load balancing, and expert specialization is not worth it relative to the model's overall size. A 3-billion-parameter dense model built for edge deployment gains little from being split into experts; the overhead of routing at that scale can cancel out much of the theoretical benefit. Mixture of experts earns its complexity at scale, not everywhere. As a rough rule of thumb, the smaller and more narrowly scoped a model is meant to be, the less likely sparse routing is to pay for itself once the extra engineering and memory overhead are counted honestly.
Scaling laws in AI research have consistently shown that bigger models, trained on more data with more compute, tend to perform better, sometimes dramatically better, on a wide range of tasks. This created enormous pressure to keep growing model size. But dense scaling has a brutal arithmetic problem: if compute cost scales in lockstep with parameter count, then at some point the cost of serving a model to real users, at real request volumes, becomes the binding constraint rather than research ambition. Mixture of experts is one of the main tools the field found to keep growing capability without hitting that wall as hard. It let labs keep following the scaling laws that had proven reliable for years, without accepting the full linear cost increase that scaling a dense model the same amount would have required.
For a business relying on a third-party model provider, this shows up indirectly but concretely. It shows up in pricing tiers that offer strong performance at a lower per-token cost than a dense model of comparable total quality would need to charge. It shows up in latency, since fewer active parameters per token generally means faster responses. And it shows up in how vendors describe their models, with terms like "active parameters" appearing alongside total parameter counts specifically because mixture of experts made that distinction meaningful in a way it never was for dense models. A buyer who never looks past the headline capability score is effectively ignoring one of the clearest signals available about why one vendor's pricing looks so different from another's for models that seem, on paper, comparably capable.
The competitive pressure among frontier labs to build ever-larger, ever-more-capable models made mixture of experts close to unavoidable at the top end of the market. A lab trying to compete on raw capability without adopting some form of sparse activation would be trying to match dense-model economics against systems that get more total knowledge per dollar of compute. That is a hard position to hold for long, which is a big part of why the technique spread as fast as it did once a handful of labs demonstrated it worked reliably at scale.
None of this means mixture of experts is a silver bullet or that bigger total parameter counts always translate into better real-world performance. Routing quality, training data, and fine-tuning still matter enormously, and a poorly trained mixture of experts model can underperform a well-trained dense model of a similar active-parameter size. The architecture changes the economics of scale; it does not automatically produce better judgment, better reasoning, or fewer mistakes. Those still depend on everything else that goes into building a good model. A buyer comparing two models with similar total and active parameter counts should still expect meaningful quality differences between them, driven by training choices that have nothing to do with the mixture of experts label either model carries.
Mixture of experts fits best in large-scale, general-purpose models meant to handle a wide variety of tasks and serve high request volumes, which is exactly the profile of the frontier models most people interact with through chat interfaces and APIs. When a model needs to be good at code, good at writing, good at math, and good at following complicated instructions all at once, giving it more total capacity through specialized experts, while keeping per-request cost manageable, is a genuinely good match for the problem.
It fits less well in narrow, single-purpose systems. If a company is building a model to do one specific job, like classifying support tickets into a fixed set of categories, the benefit of a huge pool of specialized experts mostly disappears, because the task does not need that breadth of specialization in the first place. A smaller dense model, trained specifically for that task, is usually simpler to build, cheaper to host, and just as accurate, sometimes more accurate, because all its capacity is aimed at one job rather than spread across many.
It also fits poorly in situations with tight infrastructure constraints, especially memory-constrained environments like certain edge devices or on-premise setups without generous hardware budgets. Because all experts typically need to be loaded into memory even though only a few activate per input, a mixture of experts model can demand more memory than an equally capable dense model built for that specific constraint. Teams evaluating on-device or tightly bounded deployments need to look past the "efficient compute" pitch and check the actual memory footprint.
There is also a research and interpretability angle worth being honest about. Understanding exactly why a router sent a particular token to a particular expert is harder than understanding a dense model's behavior, and the field's tools for explaining model decisions are less mature for sparse architectures. For teams in regulated industries who need to explain model behavior to auditors or regulators, this added opacity is a real cost that belongs in the decision, not an afterthought. It is worth asking a vendor directly whether their compliance and audit tooling has actually been tested against a mixture of experts model, rather than assuming the same explainability tools that work for a dense model will transfer over without gaps.
For most companies, adopting mixture of experts does not mean building one from scratch. It means choosing a vendor or open model that already uses the architecture and understanding what that choice actually buys and costs. The first practical step is to look past total parameter counts in marketing material and find the active parameter count, since that number correlates much more closely with real inference cost and latency than the headline figure does. A model advertised as having hundreds of billions of parameters might behave, cost-wise, much closer to a mid-sized dense model once you know its active count.
The second step is testing actual latency and throughput under conditions that match real usage, not just published benchmark numbers, since a routing pattern that looks efficient in a controlled benchmark can behave very differently once real, messy, unpredictable traffic starts hitting the system. Because routing behavior can vary with input type, a model's speed on one kind of task, say short customer service replies, might differ noticeably from its speed on another, say long document analysis, in ways that a single averaged benchmark score will not reveal. Running a company's own representative workload through a candidate model before committing to it catches this kind of variance early.
Teams building their own models, rather than buying access to someone else's, should budget real engineering time for the parts of mixture of experts that do not show up in a simple architecture diagram: load balancing across experts, handling uneven traffic patterns that skew which experts get exercised, and monitoring for routing collapse during training. These are solvable problems with well-documented techniques by now, but they are not free, and a team that treats mixture of experts as a drop-in upgrade to a dense architecture without planning for this will run into trouble. It also helps to plan hardware topology around expert placement early, since decisions about which experts sit on which machines can meaningfully affect latency once the system is under real production traffic rather than clean test conditions.
Finally, it is worth staying skeptical of architecture as a marketing point on its own. A model being mixture of experts is not, by itself, evidence that it is better than a comparable dense model; it is evidence about how the model's cost and capacity scale, which is a different question from how good its outputs actually are. The way to judge a model remains the same regardless of what is under the hood: test it against representative tasks, check its failure modes, and weigh cost against measured quality rather than architecture labels. A team that keeps this distinction clear will spend less time debating architecture diagrams and more time running the tests that actually predict whether a model will work for the job in front of them.
Mixture of experts is a neural network architecture that splits part of a model into multiple specialized subnetworks, called experts, and uses a learned router to send each input to only a few of them rather than processing every input through the entire network.
Companies use mixture of experts because it allows a model to have a much larger total capacity for learning and storing patterns while keeping the actual compute cost per request closer to that of a smaller dense model, which matters a great deal at high request volumes.
Not automatically. The architecture changes how compute and capacity scale together, but the quality of a model still depends heavily on training data, fine-tuning, and how well the routing was trained, so a poorly built mixture of experts model can underperform a well-built dense one.
It depends on what is being measured. Compute cost per input is often lower than a dense model of comparable total capability, but memory requirements can be higher because all experts typically need to be loaded even though only a few activate per input.
The router is a small trained component that scores every available expert for each incoming token and sends the token to the top-scoring one or two experts, learning these routing patterns during training alongside the experts themselves.
Technically yes, but it usually only makes sense at meaningful scale. For narrow, single-purpose applications, a smaller dense model is typically simpler to train, host, and debug, with comparable accuracy for that specific task.
Yes, generally for the worse. Understanding why a router sent a specific input to a specific expert is harder than tracing behavior through a dense network, which can matter for teams that need to explain model decisions to auditors or regulators.
No. The core idea has existed in machine learning research since the early 1990s, originally studied in much smaller networks built for narrower classification tasks. What changed is that it became practical and reliable to apply at the scale of today's largest transformer architecture\-based language models, which is why it became widely known recently rather than decades ago when it was first proposed.
Vendor documentation or model cards usually disclose this, often alongside both a total parameter count and an active parameter count. If a model card lists two different parameter figures, that is a strong signal it uses mixture of experts or a similar sparse activation approach worth understanding.
