# R1: FX100 Comprehensive LLM Inference and Training Benchmark (8 × AMD MI308X)

> One report, five questions: how much the LMCache parallel-read patch helps, whether KV tiering scales across GPUs, how much faster 8 GPUs load models concurrently, how effective utilization changes under frequent model switching, and how much faster concurrent training checkpoint writes become. Test models are Qwen2.5-32B and 7B.

URL: https://vuke.top/en/evidence/R1 · Updated November 12, 2026

One report, five questions: how much the LMCache parallel-read patch helps, whether KV tiering scales across GPUs, how much faster 8 GPUs load models concurrently, how effective utilization changes under frequent model switching, and how much faster concurrent training checkpoint writes become. Test models are Qwen2.5-32B and 7B.

- Report: R1
- Date: 2026-07-03
- Type: Benchmark report
- Test platform: 8 × AMD Instinct MI308X · ROCm 7.2

## Key findings

- LMCache parallel-read patch: one GPU, concurrency 16, cold read from disk; TTFT 37.97 s → 9.30 s (4.1×), read bandwidth 0.98 → 5.23 GB/s
- Training checkpoints: 65.6 GB full-model snapshots save in 94 s instead of 178 s (1.9×); sustained write bandwidth 3.26 → 6.40 GB/s
- Concurrent model loading: 8 GPUs cold-reading at once, 16% faster wall-clock and 30% faster per model
- Model switching: at 20 switches per hour, effective compute utilization rises from 46.7% to 62.8%

## Figures this report supports

| Metric | Value | Baseline | Detail | Source |
| --- | --- | --- | --- | --- |
| TTFT gain from the LMCache parallel-read patch | 4.1× | vs. before the patch | One GPU, concurrency 16, cold read from disk (Qwen2.5-32B): TTFT 37.97 s → 9.30 s; read bandwidth 0.98 → 5.23 GB/s, about 5.3×. | Measured · [R1](https://vuke.top/en/evidence/R1) |
| Faster training checkpoint saves | 1.9× | vs. the baseline in report R1 | 8-GPU 32B LoRA with 65.6 GB full-model snapshots: 178 s → 94 s; sustained write bandwidth 3.26 → 6.40 GB/s (+96%). | Measured · [R1](https://vuke.top/en/evidence/R1) |
| Effective compute utilization under model switching | 46.7% → 62.8% | at 20 model switches per hour | Modeled from measured load times: at 20 model switches per hour, effective compute utilization rises from 46.7% to 62.8%. | Measured · [R1](https://vuke.top/en/evidence/R1) |
| Concurrent model loading on 8 GPUs | 16% / 30% | vs. the baseline in report R1 | Eight GPUs cold-reading models at once: 16% faster wall-clock overall and 30% faster per model. | Measured · [R1](https://vuke.top/en/evidence/R1) |
| Arrays to inference nodes | 1 : 8 | production sizing basis | One FX100 serves 8 inference nodes; 100 GbE sustaining KV read-back for 8 nodes is a measured anchor. On the joint-test basis, storage spend stays within roughly 10% of the GPU budget. | Estimated · [R1](https://vuke.top/en/evidence/R1) [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3) |

## Test platform

- Accelerators: 8 × AMD Instinct MI308X (192 GB HBM each, gfx942)
- Host: 2 × AMD EPYC 9654 (384 threads), about 1.5 TB RAM
- Software stack: ROCm 7.2 · vLLM 0.20.1+rocm721 · LMCache (built from upstream mainline, 2026-06-29)
- Device under test: FX100 all-flash NVMe-oF array: 4-drive RAID0 (14 TB, XFS), RoCEv2, one 100 GbE port
- Baselines: A single local NVMe drive (PCIe Gen4, 2 TB); no external storage, full recompute
- 480B model: Qwen3-Coder-480B-FP8 (MoE, about 450 GB of weights)

## Request the original reports

Tell us which report IDs you need and what for, and we will reply with how to get them. The R8 code export and raw data are provided on request in the same way.

- Company: Xinyuanyi (Tianjin) Semiconductor Equipment Co., Ltd.
- Contact: Wang Limin
- Phone: +86 131 6181 8898
- Email: mingxin@agentmail.to (The inbox is monitored and triaged by AI around the clock)