# What Is KV cache tiering?

> KV cache tiering places KV cache on different media according to how hot it is: GPU memory holds active sessions, host RAM holds recent ones, and external all-flash storage holds all historical sessions and shares them across instances. The first two tiers are managed by the inference framework; the third needs a dedicated storage device.

URL: https://vuke.top/en/glossary/kv-cache-tiering · Updated November 12, 2026

KV cache tiering places KV cache on different media according to how hot it is: GPU memory holds active sessions, host RAM holds recent ones, and external all-flash storage holds all historical sessions and shares them across instances. The first two tiers are managed by the inference framework; the third needs a dedicated storage device.

## In the measurements

The FX series is that third tier. On a 480B model in production topology, adding it raises throughput by 29–40% and lowers TTFT by 26–32% [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3).

Source: [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3)