What Is NVMe-oF?

• R2FX100 KV Cache Benchmark (480B, TP8, Long Context)Qwen3-Coder-480B-FP8 deployed as one 8-GPU instance (TP=8), the standard production topology, under a long-context cold-recovery load. A three-way comparison of FX100, a single local NVMe drive and full recompute without external storage, stepped across concurrency 8 / 16 / 32. Most throughput and TTFT figures quoted on this site come from this report.2026-07-05
  8 × AMD Instinct MI308X · ROCm 7.2

• Local NVMe or an NVMe-oF Array for KV Cache?→

  Both avoid full recompute. Across three concurrency levels on a 480B model, an external FX100 array gives more throughput and lower TTFT than one local NVMe drive.

• vLLM→

  vLLM is an open-source LLM inference and serving engine known for its PagedAttention memory management and high throughput. It is a common choice for serving open-source models in production.

• RoCEv2→

  RoCEv2 (RDMA over Converged Ethernet v2) is a protocol that implements RDMA on Ethernet. Its packets are encapsulated in UDP/IP, so they can be routed across layer-3 networks. RDMA moves data directly between the memory of two machines, bypassing the operating-system kernel, which keeps latency and CPU usage low.

All terms→