# Storage Acceleration: KV Cache Tiering

> Add an FX all-flash NVMe-oF tier beyond GPU memory and host RAM, and let vLLM + LMCache write overflowing KV cache into it and read it back on a hit. On a 480B model in production topology this yields 29–40% more throughput and 26–32% lower TTFT than a single local NVMe drive.

URL: https://vuke.top/en/solutions/storage-acceleration · Updated November 12, 2026

Storage acceleration

## The problem

Long contexts, multi-turn chat and agent workloads keep pushing KV cache out of GPU memory. When an evicted session returns, the GPU has to recompute the entire context, which can push time to first token past a hundred seconds and drag throughput down.

## Our approach

Add an FX all-flash NVMe-oF tier beyond GPU memory and host RAM, and let vLLM + LMCache write overflowing KV cache into it and read it back on a hit. On a 480B model in production topology this yields 29–40% more throughput and 26–32% lower TTFT than a single local NVMe drive [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3).

## Key points and sources

- More capacity before more GPUs: on the joint-test basis storage spend stays within roughly 10% of the GPU budget and buys a 29–40% throughput gain [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3)
- Session-recovery storms stop taking the service down: 8.6–20× faster than full recompute without external storage [R2](https://vuke.top/en/evidence/R2)
- Several instances share one cache: cross-instance hot sharing through the fs:// shared pool is verified, with two independent runs within ≤5% [R3](https://vuke.top/en/evidence/R3)
- Existing clusters benefit too: on an Ascend 910B platform bottlenecked by NFS, model loading is 6.2–9.3× faster [R9](https://vuke.top/en/evidence/R9)

## Related test reports

- [R1](https://vuke.top/en/evidence/R1) · FX100 Comprehensive LLM Inference and Training Benchmark (8 × AMD MI308X)
- [R2](https://vuke.top/en/evidence/R2) · FX100 KV Cache Benchmark (480B, TP8, Long Context)
- [R3](https://vuke.top/en/evidence/R3) · FX100 KV Cache Benchmark Summary (480B, TP4×2, All Metrics)
- [R4](https://vuke.top/en/evidence/R4) · FX100 KV Cache Benchmark (480B, Multi-Instance, No. -006)
- [R9](https://vuke.top/en/evidence/R9) · FX100 on Huawei Ascend 910B: Inference and Training Benchmark (vs. NFS Baseline)

## Related products

- [FX100](https://vuke.top/en/products/fx100): The workhorse with the most complete benchmark record
- [FX200](https://vuke.top/en/products/fx200): The PCIe 4.0 model with twice the port speed and IOPS
- [FX300](https://vuke.top/en/products/fx300): The PCIe 5.0 performance tier with 6 DPUs

## Test jointly first, decide afterwards

Every engagement starts with a gated joint test of about 10 weeks. Four gates are passed in sequence; if one is missed, the project stops there, so you never pay for a benefit that was not demonstrated.

- **G1 · Week 2 · Delivery acceptance**: All materials delivered and inspected; every array passes its power-on self-test with all 24 drives recognized.
- **G2 · Week 4 · Single-node baseline**: The single-node 8-GPU inference baseline is reproduced within ±10% of the R1 figures; a local stress test of one array reaches 83% line efficiency against the measured anchor.
- **G3 · Week 7 · Main joint-test gate**: KV tiering must deliver both a TTFT reduction of ≥25% and a throughput gain inside the measured +29–40% band; with 8 nodes sharing one array, read-back bandwidth approaches the NIC ceiling.
- **G4 · Week 10 · Stability and sign-off**: 72 hours of continuous load without interruption; single-drive and single-link fault injection stays invisible to the workload; a signed joint-test report is issued.