Skip to content
Xinyuanyi

Report R3 · Benchmark summary report (official edition)

FX100 KV Cache Benchmark Summary (480B, TP4×2, All Metrics)

The same 8-GPU server redeployed as two 4-GPU instances (TP4×2), with every metric recorded across nine load levels: TTFT p50 / p90 / p99, TPOT, throughput and disk bandwidth. The report also verifies two things: the fs:// shared pool lets both instances hot-share one cache, and two independent runs at the same level deviate by no more than 5%.

Date
Type
Benchmark summary report (official edition)
Test platform
8 × AMD Instinct MI308X · ROCm 7.2

Key findings

  • Whole-machine throughput gain of +35–36% in the TP4×2 layout
  • Cross-instance hot sharing through the fs:// shared pool verified
  • Two independent runs within ≤5% of each other

Note: The signed original uses the earlier product name (AISSD5000). The edition we share is brand-unified: the product is called FX100 throughout, the layout was rebuilt and charts plus a numerical-verification appendix were added; all nine load levels match the signed original cell for cell. The signed original is available on request for cross-checking.

Figures this report supports

+29–40%

Higher inference throughput

vs. a single local NVMe drive

Qwen3-Coder-480B-FP8 in TP=8 production topology under a long-context cold-recovery load: +29% at concurrency 8 and +40% at concurrency 16, the operating point with the largest gain; +35–36% for the whole machine in the dual-instance TP4×2 layout.

MeasuredSourceR2R3

≤5%

Deviation between two independent runs

same load level, measured twice

Two independent runs at the same load level deviate by no more than 5%; cross-instance hot sharing through the fs:// shared pool was verified in the same campaign.

MeasuredSourceR3

1 : 8

Arrays to inference nodes

production sizing basis

One FX100 serves 8 inference nodes; 100 GbE sustaining KV read-back for 8 nodes is a measured anchor. On the joint-test basis, storage spend stays within roughly 10% of the GPU budget.

EstimatedSourceR1R2R3

Test platform

Applies to R1, R2, R3, R4. Other reports state their platform in their own entries.
Accelerators8 × AMD Instinct MI308X (192 GB HBM each, gfx942)
Host2 × AMD EPYC 9654 (384 threads), about 1.5 TB RAM
Software stackROCm 7.2 · vLLM 0.20.1+rocm721 · LMCache (built from upstream mainline, 2026-06-29)
Device under testFX100 all-flash NVMe-oF array: 4-drive RAID0 (14 TB, XFS), RoCEv2, one 100 GbE port
BaselinesA single local NVMe drive (PCIe Gen4, 2 TB); no external storage, full recompute
480B modelQwen3-Coder-480B-FP8 (MoE, about 450 GB of weights)

Pages that cite this report

Request the original reports

Tell us which report IDs you need and what for, and we will reply with how to get them. The R8 code export and raw data are provided on request in the same way.

Request the original reportQuote this ID when you askR3

All reports

Test jointly first, decide afterwards

Test jointly first, decide afterwards

Every engagement starts with a gated joint test of about 10 weeks. Four gates are passed in sequence; if one is missed, the project stops there, so you never pay for a benefit that was not demonstrated.

Contact
Wang Limin
Phone
+86 131 6181 8898
Email
mingxin@agentmail.to

The inbox is monitored and triaged by AI around the clock