Report R2 · Benchmark report (official edition)

FX100 KV Cache Benchmark (480B, TP8, Long Context)

Qwen3-Coder-480B-FP8 deployed as one 8-GPU instance (TP=8), the standard production topology, under a long-context cold-recovery load. A three-way comparison of FX100, a single local NVMe drive and full recompute without external storage, stepped across concurrency 8 / 16 / 32. Most throughput and TTFT figures quoted on this site come from this report.

Date: July 5, 2026
Type: Benchmark report (official edition)
Test platform: 8 × AMD Instinct MI308X · ROCm 7.2

Key findings

Throughput: 56.6 vs 43.9 tok/s at concurrency 8 (+29%); 74.9 vs 53.6 tok/s at concurrency 16 (+40%)
TTFT p50: 7.53 / 11.85 / 26.35 s across the three levels, against 10.17 / 17.31 / 35.73 s on a single local NVMe drive, a 26–32% reduction
Against full recompute without external storage (concurrency 16): TTFT p50 149.48 s → 11.85 s, throughput 4.1 → 74.9 tok/s

Raw readings

FX100 all-flash arraySingle local NVMe drive

TTFT p50 (s)

↓ lower is better

Concurrency8

Local NVMe drive · Concurrency 8: 10.17 s
FX100 · Concurrency 8: 7.53 s

−26%

Concurrency16

Local NVMe drive · Concurrency 16: 17.31 s
FX100 · Concurrency 16: 11.85 s

−32%

Concurrency32

Local NVMe drive · Concurrency 32: 35.73 s
FX100 · Concurrency 32: 26.35 s

−26%

Throughput (tok/s)

↑ higher is better

Concurrency8

Local NVMe drive · Concurrency 8: 43.9 tok/s
FX100 · Concurrency 8: 56.6 tok/s

+29%

Concurrency16

Local NVMe drive · Concurrency 16: 53.6 tok/s
FX100 · Concurrency 16: 74.9 tok/s

+40%

Concurrency32

Local NVMe drive · Concurrency 32: 53.9 tok/s
FX100 · Concurrency 32: 71.6 tok/s

+33%

Fig. 2 · Qwen3-Coder-480B-FP8 · TP=8 · 8 × MI308X · long-context cold-recovery load. TTFT is p50 in seconds; throughput in tok/s; full recompute was measured at concurrency 16 only.R2

Table 1 · Qwen3-Coder-480B-FP8 · TP=8 · 8 × MI308X · long-context cold-recovery load. TTFT is p50 in seconds; throughput in tok/s; full recompute was measured at concurrency 16 only. Source: R2
Concurrency	FX100 TTFT	Local NVMe drive TTFT	Full recompute TTFT	FX100 tok/s	Local NVMe drive tok/s	Full recompute tok/s
8	7.53 s	10.17 s	—	56.6	43.9	—
16	11.85 s	17.31 s	149.48 s	74.9	53.6	4.1
32	26.35 s	35.73 s	—	71.6	53.9	—

Figures this report supports

+29–40%

Higher inference throughput

vs. a single local NVMe drive

Qwen3-Coder-480B-FP8 in TP=8 production topology under a long-context cold-recovery load: +29% at concurrency 8 and +40% at concurrency 16, the operating point with the largest gain; +35–36% for the whole machine in the dual-instance TP4×2 layout.

MeasuredSourceR2 R3

−26–32%

Lower time to first token (TTFT)

vs. a single local NVMe drive

Across three concurrency levels of the same 480B TP8 load, TTFT p50 falls from 10.17–35.73 s to 7.53–26.35 s.

MeasuredSourceR2

8.6–20×

Faster than recomputing without external storage

vs. full recompute with no external storage

At concurrency 16, TTFT p50 is 149.5 s when recomputing and 11.85 s with FX100; throughput is 4.1 tok/s versus 74.9 tok/s.

MeasuredSourceR2

1 : 8

Arrays to inference nodes

production sizing basis

One FX100 serves 8 inference nodes; 100 GbE sustaining KV read-back for 8 nodes is a measured anchor. On the joint-test basis, storage spend stays within roughly 10% of the GPU budget.

EstimatedSourceR1 R2 R3

Test platform

Applies to R1, R2, R3, R4. Other reports state their platform in their own entries.
Accelerators	8 × AMD Instinct MI308X (192 GB HBM each, gfx942)
Host	2 × AMD EPYC 9654 (384 threads), about 1.5 TB RAM
Software stack	ROCm 7.2 · vLLM 0.20.1+rocm721 · LMCache (built from upstream mainline, 2026-06-29)
Device under test	FX100 all-flash NVMe-oF array: 4-drive RAID0 (14 TB, XFS), RoCEv2, one 100 GbE port
Baselines	A single local NVMe drive (PCIe Gen4, 2 TB); no external storage, full recompute
480B model	Qwen3-Coder-480B-FP8 (MoE, about 450 GB of weights)

Pages that cite this report

Product: FX100
The workhorse with the most complete benchmark record
Solution: Storage Acceleration: KV Cache Tiering
Storage acceleration
Solution: AI Datacenter Construction
Datacenter construction

Request the original reports

Tell us which report IDs you need and what for, and we will reply with how to get them. The R8 code export and raw data are provided on request in the same way.

Request the original reportQuote this ID when you askR2

All reports

Test jointly first, decide afterwards

Every engagement starts with a gated joint test of about 10 weeks. Four gates are passed in sequence; if one is missed, the project stops there, so you never pay for a benefit that was not demonstrated.

Book a joint test See the evidence

Contact: Wang Limin
Phone: +86 131 6181 8898
Email: mingxin@agentmail.to
The inbox is monitored and triaged by AI around the clock