# Recompute or Read Back? KV Cache on a 480B Model

> Full recompute versus reading KV cache back from FX100 on a 480B model at concurrency 16: time to first token, throughput, test conditions and how to reproduce.

URL: https://vuke.top/en/insights/recompute-vs-read-back-480b · Updated November 12, 2026

Published: 2026-11-12

**Question**: After KV cache is evicted from GPU memory, is it faster to recompute or to read it back from external storage?

**Short answer**: Reading back is far faster. For Qwen3-Coder-480B-FP8 under a long-context cold-recovery load at concurrency 16, full recompute without external storage gives a TTFT of 149.48 s at p50 and 4.1 tok/s; keeping the KV cache on FX100 and reading it back gives 11.85 s and 74.9 tok/s [R2](https://vuke.top/en/evidence/R2).

## How the test was run

The model is Qwen3-Coder-480B-FP8 deployed as one 8-GPU instance (TP=8), the production topology, on 8 × AMD MI308X. The load is long-context cold recovery: a session's KV cache is no longer in GPU memory, so an incoming request must either recompute it or read it back.

Three setups were compared under the same load: the FX100 all-flash array, a single local NVMe drive, and full recompute without external storage. FX100 and the local drive were measured at concurrency 8, 16 and 32; full recompute was measured at concurrency 16 only [R2](https://vuke.top/en/evidence/R2).

## Three readings at concurrency 16

TTFT at p50 and throughput for the three setups at the same concurrency.

- Full recompute, no external storage: TTFT 149.48 s at p50, 4.1 tok/s [R2](https://vuke.top/en/evidence/R2).
- Single local NVMe drive: TTFT 17.31 s at p50, 53.6 tok/s [R2](https://vuke.top/en/evidence/R2).
- FX100 all-flash array: TTFT 11.85 s at p50, 74.9 tok/s [R2](https://vuke.top/en/evidence/R2).

## Two baselines that must not be mixed

A speed-up depends on what you compare with. This test has two denominators.

Against full recompute without external storage, the speed-up is 8.6–20× [R2](https://vuke.top/en/evidence/R2). That figure answers whether the KV cache should be kept at all.

Against a single local NVMe drive, throughput rises by 29–40% and TTFT at p50 falls by 26–32% [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3). That figure answers where to keep it. Folding the two into one sentence would suggest that an external array is an order of magnitude faster than a local drive, which is not what was measured.

## Whether the figures can be reproduced

Two independent runs at the same load level deviate by no more than 5% [R3](https://vuke.top/en/evidence/R3). The conclusions for both deployment topologies were compiled into an official report by a third-party testing body [R4](https://vuke.top/en/evidence/R4).

The LMCache parallel-read patch, the load client, the orchestration scripts and the raw data are packaged as R8 and can be rerun on your own cluster [R8](https://vuke.top/en/evidence/R8). Your model, context length and concurrency differ, and so will your readings; this data shows the order of magnitude and the direction.

## Related test reports

- [R2](https://vuke.top/en/evidence/R2) · FX100 KV Cache Benchmark (480B, TP8, Long Context) (8 × AMD Instinct MI308X · ROCm 7.2)
- [R3](https://vuke.top/en/evidence/R3) · FX100 KV Cache Benchmark Summary (480B, TP4×2, All Metrics) (8 × AMD Instinct MI308X · ROCm 7.2)
- [R4](https://vuke.top/en/evidence/R4) · FX100 KV Cache Benchmark (480B, Multi-Instance, No. -006) (8 × AMD Instinct MI308X · ROCm 7.2)
- [R8](https://vuke.top/en/evidence/R8) · FX100 KV Cache Test Code Export and Raw Benchmark Data (AMD Instinct MI308X · ROCm 7.2)

## External references

- [LMCache (GitHub repository)](https://github.com/LMCache/LMCache) · LMCache Project. LMCache source code, architecture and storage backends
- [vLLM Documentation](https://docs.vllm.ai/) · vLLM Project. Official documentation of vLLM deployment, prefix caching and tensor-parallel settings