What Is TP (tensor parallelism)?

• R2FX100 KV Cache Benchmark (480B, TP8, Long Context)Qwen3-Coder-480B-FP8 deployed as one 8-GPU instance (TP=8), the standard production topology, under a long-context cold-recovery load. A three-way comparison of FX100, a single local NVMe drive and full recompute without external storage, stepped across concurrency 8 / 16 / 32. Most throughput and TTFT figures quoted on this site come from this report.2026-07-05
  8 × AMD Instinct MI308X · ROCm 7.2
• R3FX100 KV Cache Benchmark Summary (480B, TP4×2, All Metrics)The same 8-GPU server redeployed as two 4-GPU instances (TP4×2), with every metric recorded across nine load levels: TTFT p50 / p90 / p99, TPOT, throughput and disk bandwidth. The report also verifies two things: the fs:// shared pool lets both instances hot-share one cache, and two independent runs at the same level deviate by no more than 5%.2026-07-06
  8 × AMD Instinct MI308X · ROCm 7.2

• HBM (high-bandwidth memory)→

  HBM (high-bandwidth memory) is the stacked, high-bandwidth memory used on AI accelerators, in other words GPU memory. It is extremely fast but limited in capacity and expensive, and model weights and KV cache compete for it.

• MoE (mixture of experts)→

  MoE (mixture of experts) is a model architecture in which each layer contains several expert sub-networks and each token activates only a few of them. It lets a model grow very large without a proportional increase in compute per token, at the cost of a very large set of weights.

All terms→