# What Is Prefix caching?

> Prefix caching is the mechanism by which an inference engine reuses the KV cache of an identical input prefix: when two requests start the same way (the same system prompt or the same document), the second can skip computing that part. The prefix caching built into vLLM works only within GPU memory.

URL: https://vuke.top/en/glossary/prefix-caching · Updated November 12, 2026

Prefix caching is the mechanism by which an inference engine reuses the KV cache of an identical input prefix: when two requests start the same way (the same system prompt or the same document), the second can skip computing that part. The prefix caching built into vLLM works only within GPU memory.

## In the measurements

An external KV cache extends the mechanism beyond GPU memory: evicted prefixes can still be hit, and they can be shared across instances [R3](https://vuke.top/en/evidence/R3).

Source: [R3](https://vuke.top/en/evidence/R3)