# What Is MoE (mixture of experts)?

> MoE (mixture of experts) is a model architecture in which each layer contains several expert sub-networks and each token activates only a few of them. It lets a model grow very large without a proportional increase in compute per token, at the cost of a very large set of weights.

URL: https://vuke.top/en/glossary/moe · Updated November 12, 2026

MoE (mixture of experts) is a model architecture in which each layer contains several expert sub-networks and each token activates only a few of them. It lets a model grow very large without a proportional increase in compute per token, at the cost of a very large set of weights.

## In the measurements

Qwen3-Coder-480B-FP8, the model in the 480B benchmarks on this site, is an MoE model with about 450 GB of weights [R2](https://vuke.top/en/evidence/R2).

Source: [R2](https://vuke.top/en/evidence/R2)