Glossary
What Is MoE (mixture of experts)?
MoE (mixture of experts) is a model architecture in which each layer contains several expert sub-networks and each token activates only a few of them. It lets a model grow very large without a proportional increase in compute per token, at the cost of a very large set of weights.
Updated
In the measurements
Qwen3-Coder-480B-FP8, the model in the 480B benchmarks on this site, is an MoE model with about 450 GB of weightsR2.
Related test reports
Nearby terms
- TP (tensor parallelism)
Tensor parallelism (TP) splits every layer of a model across several GPUs that compute it together. TP=8 means one model instance spans 8 GPUs, a common way to deploy models too large for a single card; TP4×2 divides 8 GPUs into two instances of 4 GPUs each.
- ROCm
ROCm is AMD's open-source software stack for GPU computing, covering drivers, compilers, math libraries and deep-learning framework support. On AMD Instinct accelerators it plays the role CUDA plays on NVIDIA platforms.
Test jointly first, decide afterwards
Test jointly first, decide afterwards
Every engagement starts with a gated joint test of about 10 weeks. Four gates are passed in sequence; if one is missed, the project stops there, so you never pay for a benefit that was not demonstrated.
- Contact
- Wang Limin
- Phone
- +86 131 6181 8898
- mingxin@agentmail.to
The inbox is monitored and triaged by AI around the clock