# R2: FX100 KV Cache Benchmark (480B, TP8, Long Context)

> Qwen3-Coder-480B-FP8 deployed as one 8-GPU instance (TP=8), the standard production topology, under a long-context cold-recovery load. A three-way comparison of FX100, a single local NVMe drive and full recompute without external storage, stepped across concurrency 8 / 16 / 32. Most throughput and TTFT figures quoted on this site come from this report.

URL: https://vuke.top/en/evidence/R2 · Updated November 12, 2026

Qwen3-Coder-480B-FP8 deployed as one 8-GPU instance (TP=8), the standard production topology, under a long-context cold-recovery load. A three-way comparison of FX100, a single local NVMe drive and full recompute without external storage, stepped across concurrency 8 / 16 / 32. Most throughput and TTFT figures quoted on this site come from this report.

- Report: R2
- Date: 2026-07-05
- Type: Benchmark report (official edition)
- Test platform: 8 × AMD Instinct MI308X · ROCm 7.2

## Key findings

- Throughput: 56.6 vs 43.9 tok/s at concurrency 8 (+29%); 74.9 vs 53.6 tok/s at concurrency 16 (+40%)
- TTFT p50: 7.53 / 11.85 / 26.35 s across the three levels, against 10.17 / 17.31 / 35.73 s on a single local NVMe drive, a 26–32% reduction
- Against full recompute without external storage (concurrency 16): TTFT p50 149.48 s → 11.85 s, throughput 4.1 → 74.9 tok/s

## Raw readings

| Concurrency | FX100 TTFT p50 (s) | Local NVMe TTFT p50 (s) | Full recompute TTFT p50 (s) | FX100 tok/s | Local NVMe tok/s | Full recompute tok/s |
| --- | --- | --- | --- | --- | --- | --- |
| 8 | 7.53 | 10.17 | — | 56.6 | 43.9 | — |
| 16 | 11.85 | 17.31 | 149.48 | 74.9 | 53.6 | 4.1 |
| 32 | 26.35 | 35.73 | — | 71.6 | 53.9 | — |

Qwen3-Coder-480B-FP8 · TP=8 · 8 × MI308X · long-context cold-recovery load. TTFT is p50 in seconds; throughput in tok/s; full recompute was measured at concurrency 16 only. Source: [R2](https://vuke.top/en/evidence/R2)

## Figures this report supports

| Metric | Value | Baseline | Detail | Source |
| --- | --- | --- | --- | --- |
| Higher inference throughput | +29–40% | vs. a single local NVMe drive | Qwen3-Coder-480B-FP8 in TP=8 production topology under a long-context cold-recovery load: +29% at concurrency 8 and +40% at concurrency 16, the operating point with the largest gain; +35–36% for the whole machine in the dual-instance TP4×2 layout. | Measured · [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3) |
| Lower time to first token (TTFT) | −26–32% | vs. a single local NVMe drive | Across three concurrency levels of the same 480B TP8 load, TTFT p50 falls from 10.17–35.73 s to 7.53–26.35 s. | Measured · [R2](https://vuke.top/en/evidence/R2) |
| Faster than recomputing without external storage | 8.6–20× | vs. full recompute with no external storage | At concurrency 16, TTFT p50 is 149.5 s when recomputing and 11.85 s with FX100; throughput is 4.1 tok/s versus 74.9 tok/s. | Measured · [R2](https://vuke.top/en/evidence/R2) |
| Arrays to inference nodes | 1 : 8 | production sizing basis | One FX100 serves 8 inference nodes; 100 GbE sustaining KV read-back for 8 nodes is a measured anchor. On the joint-test basis, storage spend stays within roughly 10% of the GPU budget. | Estimated · [R1](https://vuke.top/en/evidence/R1) [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3) |

## Test platform

- Accelerators: 8 × AMD Instinct MI308X (192 GB HBM each, gfx942)
- Host: 2 × AMD EPYC 9654 (384 threads), about 1.5 TB RAM
- Software stack: ROCm 7.2 · vLLM 0.20.1+rocm721 · LMCache (built from upstream mainline, 2026-06-29)
- Device under test: FX100 all-flash NVMe-oF array: 4-drive RAID0 (14 TB, XFS), RoCEv2, one 100 GbE port
- Baselines: A single local NVMe drive (PCIe Gen4, 2 TB); no external storage, full recompute
- 480B model: Qwen3-Coder-480B-FP8 (MoE, about 450 GB of weights)

## Request the original reports

Tell us which report IDs you need and what for, and we will reply with how to get them. The R8 code export and raw data are provided on request in the same way.

- Company: Xinyuanyi (Tianjin) Semiconductor Equipment Co., Ltd.
- Contact: Wang Limin
- Phone: +86 131 6181 8898
- Email: mingxin@agentmail.to (The inbox is monitored and triaged by AI around the clock)